➡ ⭐ ⭐ ⭐ 💡 ❗ 😮 😀 😎 hi all. Terence Tao, who is regarded as one of the greatest living mathematicians although of course not by himself (he does not even have a picture on his “about” page), has proved the Erdos Discrepancy Problem, fresh off a profile by the NYT.[a2] Tao has the distinction as being one of the very few mathematicians who has won the $3M breakthrough award, and also has said virtually nothing about it publicly (except at the awards ceremony). he’s too busy blogging about math ofc!
the EDP is a very interesting problem that relates in some sense to patterns in random walks. functions related to primes turn out to be basically very similar to “pseudo random walks”. this has been known for decades and there is a relation to the Riemann conjecture documented by Good/ Churchhouse in 1968.[g2]
the connection of hard math/ CS problems, and automated theorem proving, with analyzing random walks is a thread Ive been pursing myself over the last few years in my Collatz conjecture research. this gives me a nice moment/ opportunity/ milestone to write it all up.
its also another huge feather in the cap of the polymath project which was peripherally involved in the solution and a big triumph for cyber-collaborative mathematics.[b] a now-near-legendary almost-offhand musing in a blog comment led Tao in the right direction. “the devil is in the details…” 😈
Kalai is already casting around for the next “Big Kahuna”. any ideas? [b14] 😉 😀
this blog previously covered the Konev/ Lisitsa breakthrough using SAT theorem proving against EDP which got significant media coverage and ties in substantially with another key interest of this blog, empirical math/ (T)CS research.[h] so far Tao has not received mainstream media coverage—on this particular problem that is!—but my thinking is its merely because Stanford seems not yet to have published any press release on the preprint. its utterly fantastic this 8-decade old problem was solved, and maybe this will open/ pave the way for some other similar problems.
it is not yet well realized but (pseudo-) random walks seem to have deep unexplored connections to fractal theory.[c] have uncovered this in my own collatz research & think it is a major research area or research program.
which reminds me, have long been waiting for a moment to point this out. think ref [c8] by Walsh is actually one of the greatest unheralded discoveries in TCS if you ask me and (“small world”!) ties into my favorite problem P vs NP. p3:
Figure 4 suggests an interesting analogy with statistical mechanics. At the phase boundary in physical systems, problems tend to be “self-similar”. That is, they look similar at every length scale. At the phase boundary in computational systems, problems also display a form of self-similarity. Branching decisions give subproblems that look neither more or less constrained. This helps to explain why such problems are difficult to solve. Branching decisions tell us very little about the problem, giving subproblems that are neither more obviously soluble nor more obviously insoluble. We will often have to search to a large depth either for a solution or for a refutation.
this ref therefore talks about self-similarity in NP complete problems but without a single reference to “fractals”. virtually no other research has built on this much since then (although [c4][c5] take some stabs at it). is there some kind of stigma or blind spot there or what? do people think they only show up in nature or the stock market?
but there in a nutshell may be the explanation of hardness for P vs NP, an observation of monumental significance and which almost nobody else has remarked on much since its discovery: hard problems lead to subproblems that are self similar in a fractal way. formalizing this somehow is the massive question/ challenge that could lead to a proof. the study of computational hardness seems to be intrinsically/ inextricably/ tightly tied/ intertwined with randomness. this has been known for decades eg also with Razborov/ Rudich Natural proofs (also recently blogged on); however this insight seems difficult to exploit.
elsewhere, the Dyson conjecture seems to be related to behavior of random walks and also is loosely reminiscent of the collatz conjecture.[d]
both the Dyson & collatz conjecture relate to the behavior of base-n representations of repeated “powering”.[e] exponentiation dynamics/ complexity shows up in many deep areas of math eg RSA encryption.
also, entropy/ pseudorandomness are both loosely/ deeply connected in ways not fully appreciated.[f]
about the only article that seems to make some of these deep connections is a standout one by Wigderson, its a gem that seems to be published only on his own site. he starts out by remarking on the Riemann hypothesis and draws many other connections.[g]
there is a large area in TCS known as “derandomization of algorithms” and its another big piece of this puzzle. derandomization can be seen as a sort of decryption or extraction of algorithmic pseudorandomness. one of the biggest breakthroughs in the area was the AKS primality algorithm. other theoretical breakthroughs in TCS can be seen as derandomization applications and its connected to major open problems in the area. another significant area is with expander graphs as in this paper Entropy waves, the zig-zag graph product, and new constant-degree by Reingold, Vadhan, and Wigderson. from the abstract, shortly later used to great effect in the SL = L theorem by Reingold:
Crucial to our intuition (and simple analysis) of the properties of this graph product is the view of expanders as functions which act as “entropy wave” propagators — they transform probability distributions in which entropy is concentrated in one area to distributions where that concentration is dissipated. In these terms, the graph products affords the constructive interference of two such waves.
could write volumes more but am just a blogger, not a professional writer! hope that others chase down some of these connections/ leads and maybe the EDP fanfare will help increase visibility of this general area and the tricky crosscutting connections around it. am also expecting to see major mainstream media attention in the near future.
(see also earlier blog Erdös100—tribute to a brilliant contrarian)
one tiny fineprint quibble: am just as excited as or even more than everyone else—as ofc one can tell from all the emoticons—but wonder, who else in the world has actually/ really verified this extremely technical proof? it is after all a preprint… gowers seems to say its ok but did he go through it line-by-line?[b4] its surely a real life scenario replay of the old newspaper quote (difficult to trace/ attribute exactly) about einsteins theory of relativity that “only maybe a half dozen experts in the world can understand it.” its an interesting case study of the sociology of acceptance of proofs! and surely “because Tao said so” is not a definitive reason, right? must fairly note clearly/ obviously reputation is playing some role here, but thats another study for a mathematical sociologist!
- a. EDP
- b. polymath
- c. fractal
- d. dyson conjecture
- e. powering
- f. entropy/ randomness
- g. articles/ papers
- h. EDP (Konev/ Lisitsa)
- 1. Terence Tao’s Answer to the Erdős Discrepancy Problem | Quanta Magazine
- 2. The Singular Mind of Terry Tao – The New York Times
- 3. [1509.05363] The Erdos discrepancy problem
- 4. EDP28 — problem solved by Terence Tao! | Gowers’s Weblog
- 5. Frogs and Lily Pads and Discrepancy | Gödel’s Lost Letter and P=NP
- 6. Crowds beat computers in answer to Wikipedia-sized maths problem | New Scientist
- 7. The Erdos discrepancy problem via the Elliott conjecture / Tao blog
- 8. The logarithmically averaged Chowla and Elliott conjectures for two-point correlations; the Erdos discrepancy problem / Tao blog
- 9. The Mozart of Maths/ Sydney Morning Herald
- 10. Maths whizz solves a master’s riddle / Nature
- 11. EDP Reflections and Celebrations/ Kalai blog
- 1. Polymath Project – Wikipedia, the free encyclopedia
- 2. Polymath Proposals — Blogs, Pictures, and more on WordPress
- 3. SIAM: Massively Collaborative Mathematics
- 4. Main Page – Polymath1Wiki
- 5. Mathematics by collaboration | Science News
- 6. Problem Solved, LOL: A Complex Tic-Tac-Toe Puzzle Falls Thanks to Blog Comments: Scientific American
- 7. The Polymath Project: Lessons from a Successful Online Collaboration in Mathematics by Cranshaw & Kittur
- 8. Polymath1 and the Modalities of ‘Massively Collaborative Mathematics’ by Barany
- 9. Local vs. Global MO in relation to “Polymath” efforts – MathOverflow Meta
- 10. Polymath projects on StackExchange/mathoverflow? | Secret Blogging Seminar
- 11. THE “BOUNDED GAPS BETWEEN PRIMES” POLYMATH PROJECT – A RETROSPECTIVE / D.H.J. POLYMATH
- 12. Open Collaborative Mathematics over the Internet – Three Examples | Combinatorics and more
- 13. ProofWiki
- 14. big list – Proposals for polymath projects – MathOverflow
- 15. Mathematical Advances: lone or massively collaborative endeavors?
- 16. The Erdős discrepancy problem – Polymath1Wiki
- 1. Avoiding Monsters and Non-Monsters | Gödel’s Lost Letter and P=NP
- 2. reference request – What is some important work in fractals? – Theoretical Computer Science Stack Exchange
- 3. cc.complexity theory – Algorithms as fractals – Theoretical Computer Science Stack Exchange
- 4. Fractal dimension versus computational complexity
- 5. Heavy-Tailed Phenomena in Satisﬁability and Constraint Satisfaction Problems
- 6. Scale-free network – Wikipedia, the free encyclopedia
- 7. Small-world network – Wikipedia, the free encyclopedia
- 8. CiteSeerX — The Constrainedness Knife-Edge
- 1. More on Testing Dyson’s Conjecture | Gödel’s Lost Letter and P=NP
- 2. A Challenge From Dyson | Gödel’s Lost Letter and P=NP
- 3. Dyson Statements that Are Likely to Be True but Unprovable / C. S. Calude
- 4. number theory – Is there a power of 2 that, written backward, is a power of 5? – Mathematics Stack Exchange
- 1. When do n and 2n have the same decimal digits? « Stack Exchange Mathematics Blog
- 2. puzzle – Regularities when $n$ and $2n$ contain the same digits – Mathematics Stack Exchange
- 3. discrete mathematics – Interview Question Asked In yahoo – Mathematics Stack Exchange
- 4. nt.number theory – Do the base 3 digits of $2^n$ avoid the digit 2 infinitely often — what is the status of this problem? – MathOverflow
- 1. Entropy and Information Theory / Gray
- 2. information theory – What is the computer science definition of entropy? – Stack Overflow
- 3. Is there a connection between the Halting Theorem on Turing Machines and Thermodynamic Entropy? – Computer Science Stack Exchange
- 4. simulation – What randomness really is – Computer Science Stack Exchange
- 5. algorithms – proof of convergence in arbitrary precision PRNGs – Computer Science Stack Exchange
- 6. reference request – Are there pseudorandom number generators (PRNG) with no finite period? – Computer Science Stack Exchange
- 7. reference request – analyzing moving pseudorandom sieve algorithm – Computer Science Stack Exchange
- 1. COMPUTATIONAL INTRACTABILITY AND PSEUDORANDOMNESS / Wigderson
- 2. The Riemann Hypothesis and Pseudorandom Features of the Möbius Sequence / Good, Churchhouse
- 3. The Quest for Randomness » American Scientist
- 1. Computer cracks Erdős puzzle – but no human brain can check the answer : compsci
- 2. University of Liverpool – Research
- 3. Mathematicians think like machines for perfect proofs – physics-math – 25 June 2013 – New Scientist
- 4. A gentle introduction to the 5th Polymath project | The Number Warrior
- 5. reference request – what is the Erdos Discrepancy Problem connection(s) to (T)CS? – Theoretical Computer Science Stack Exchange
- 6. If no human can check a proof of a theorem, does it really count as mathematics? That’s the intriguing question raised by the latest computer-assisted proof. It is as large as the entire content of Wikipedia, making it unlikely that will ever be checked b
- 7. New Wikipedia sized proof explained with a puzzle – YouTube
- 8. Practically P=NP? | Gödel’s Lost Letter and P=NP