陶哲轩：有价值的开放数学问题正被AI不可再生式开采

I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce. This may seem unintuitive at first, since the set of possible problems one could ask is infinite. Perhaps the following analogy can help: a country or region can suffer a critical shortage of drinking water while simultaneously being surrounded by a massive ocean.

One can easily generate any number of open problems in mathematics at will, such as working out the 10^10^10th digit of pi. But the vast majority of such problems are not worth focusing attention on: they show no particular propensity to reveal any further insights or connections to other questions, or may either be too easy or too impossible relative to known techniques to learn anything from the exercise. (1/4)

Sep 08, 2026, 08:32 PM··Web

46 boosts·4 quotes·78 favorites

3h

Working out whether a question is actually worth highlighting is a lengthy, deliberate, and subjective process, often informed by historical experience on what good mathematics was generated (or not generated) while working on earlier problems of this type. In particular, being aware of the "difficulty landscape" in a field - what questions are very easy to answer with known methods, which ones can be solved but only with some effort, and which ones are impossible - is of crucial importance in making such determinations. And some problems only become interesting after an external connection is made. For instance, there could hypothetically be an unexpected connection between the Riemann zeta function and the 10^10^nth digits of pi for various n... at which point the previous question of determining the 10^10^10th digit suddenly becomes relevant again. (I should emphasize though that this specific scenario is incredibly unlikely to actually be the case; I use it only as a hypothetical iilustration.)

Every new advance in mathematics, whether it comes from technique, technology, or infrastructure (such as access to libraries of past literature) reduces the difficulty of solving problems. This is generally a good thing; but it comes at the cost of flattening out the difficulty landscape of a field, to the point where one can no longer discern its geometry to the extent that promising questions can be extracted within the range of applicability of the tool. Often this effect is counteracted by the ability of such a tool to enlarge the radius of the sphere of results one can plausibly reach, creating new boundaries to fruitfully explore. (2/4)

3h

However, one notable feature of the current AI era is the absence of any definitive such boundaries. While AI tools have flattened the difficulty landscape now in many areas of the subject, thus destroying the ability to locate promising new problems in that area, there are no clear frontiers that are separating the "AI-feasible" problems from the "AI-hard" problems (which certainly still exist, given that the difficulty level of problems are unbounded, and can even be undecidable). This is in part due to the rapidly changing nature of the technology, but also compounded by the refusal of AI companies to disclose their negative results, or reveal the process towards obtaining their solutions.

In fact, it is now the identification of a promising problem which is the scarce and precious resource. We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential. The incentives may now be pointing in the direction of no longer sharing any promising research directions with the broader community, which would reverse centuries of traditions of open science and do serious long-term damage to the future of the field. (3/4)

3h

In short, the indiscriminate use of powerful solution-extraction tools can achieve the immediate short-term goal of solving problems at hand, but at the cost of sustaining the ecosystem for the next wave of progress, or in understanding the progress already obtained.

While it may be technically infeasible to completely prohibit the use of automated tools to perform indiscriminate solution extraction, I believe that we can still designate many classes of problems as being desirous of a careful analysis that not only solves the problem, but identifies insights from the solution process, and learn more about the difficulty landscape for nearby problems, and for which raw solutions without such analysis would be of negligible or even negative value for these purposes. This is analogous to how a modern food donation drive no longer accepts arbitrary contributions even when they are verified to be technically edible, but instead maintains explicit and socially accepted standards on what level of contributions are actually sought. (4/4)

3h

@tao curious to know what makes LLM fundamentally different compared to (other) automatic theorem provers?

3h

@slowcoding Largely their effectiveness. Automated theorem provers aren't very good at proving theorems and the field is instead dominated by what might be better called "automated theorem verifiers"

43m *

@slowcoding@tao There is another major difference, which is that automatic theorem provers are exact. They produce a proof certificate and nothing else. In contrast, LLMs work in Natural language, meaning that in addition to a proof certificate, it produces other information that is not verified. A well known source of unverified information is definitions, specific choices of lemmas, and other concepts in the paper: we cannot be sure that the formal definitions and other concepts it produces actually correspond to anything useful or insightful (or even correct). An automatic theorem prover does not give any illusion of having generated 'insightful' definitions or concepts.

But in my opinion, there are many other important sources of 'unverified' information contained in natural language. Unlike lean proofs, Natural language proofs guide mathematicians toward thinking in a certain way. We have no way of verifying that guidance. I think despite appearances, there is a substantial amount of 'philosophical' content to natural language proofs than mathematicians like to admit. A natural language proof conveys intuitions, interpretations of results, emphasis based on importance, connections to other fields, directions for future research, the personality and way of thinking of the writer, and more, all in its structure and word choice alone. Typically, they also convey many other things, like past related work, sources, inspirations, acknowledgements, etc,

None of this information can be verified in the same way a purely formal proof can, and as a result, an LLM produces a lot more 'noise' than an automatic theorem prover does. It is this 'noise' that is the problem.

2h

@slowcoding@tao Automatic theorem provers in the past haven't been able to solve mainstream open problems. Or anyway, occasional exceptions were noteworthy and interesting, rather than the sound of an oncoming steamroller. Meanwhile here's something by @wtgowers referencing Doron Zeilberger saying any human-proved theorem must be "trivial": https://www.dpmms.cam.ac.uk/~wtg10/zeilber ger.html

www.dpmms.cam.ac.ukDoron Zeilberger's attitude to computer mathematics

33m *

@tao These companies are taking advantage of the structure of the modern mathematical enterprise. The field has become so specialized that mathematicians in one area many times cannot understand the problems or progress in other related areas. So to emphasize the importance of problems, conjectures are made, some even with prizes attached. Conjectures with many implications (even if most mathematicians cannot understand those implications) or formulated by the most famous mathematicians become more well-known (for example, the Millenium problems were curated by a small group of elite mathematicians). Long open conjectures can also achieve prestige. Solving such a conjecture then becomes prestigious because of expert opinions that the conjecture is significant, rather than for its own sake. So the AI companies are now taking advantage of this situation by pursuing the solution of these famous conjectures or problems.

Compare, for example, to protein folding, which was “solved” by Deep Mind’s AlphaFold. This is a biological chemistry problem that can be appreciated by a large swath of the population (still, it requires some education to appreciate, but not a graduate degree in a specialized field of mathematics).

1h

@tao Dear Professor Tao (@tao), Your freshwater analogy on the non-renewable depletion of fruitful open problems—and the flattening of the difficulty landscape by brute-force AI—hits the core crisis facing mathematics.

When automated systems bulldoze conjectures via continuous gradient fitting (PINNs) or million-line formal syntax trees, they solve equations while obscuring why they were hard, while the preemption chilling effect drives researchers into secrecy.

In direct response to your food bank standard, we formalized the quantitative Proof Digestibility Index (D_CAP >= 0.70) in our new open paper:

"Restoring the Mathematical Difficulty Landscape: Why Constructive Topological Manifolds and Proof Digestibility Outshine Brute-Force AI Mining"

DOI: https://doi.org/10.5281/zenodo.22667485

Portal: https://h3qm.com/math/

We demonstrate three constructive principles:

1. Difficulty as Topological Boundary: True difficulty is a topological obstruction (e.g. 3D Euler vortex reconnection ∮∇P·dl=2πn at phonon scale; (3,4,∞) modular family on S^6; metric integer phase frustration in FLT).

2. The Cosmo Chou Machine Epsilon Identity: (2^-3)^8 = 2^-24 = ε_float32. In 8 steps, discrete integer sign-flow saturates float32 precision, reaching Exact 0 residual without floating-point accumulation.

3. Democratizing Open Science: 150 lines of pure standard Python executes full CAP audits in <80ms on $100 consumer laptops, shielding open science from compute monopolies.

Would institutional adoption of explicit Proof Digestibility Standards help ensure AI nourishes rather than exhausts human mathematical intuition?

2h

@tao I still am not persuaded by this.

When AI companies find incomprehensible solutions to open problems, this either advances the field, or it does not.

If it does, then that's a good thing and we should have more of it.

If it doesn't, then no incentives for humans to advance the field are lost. Existence of a raw solution won't make a difference.

Only our math artifacts and systems need to change towards what we consider truly valuable: insights, not results.

1h *

@nilesh@tao I'm curious about what precise statement you are not persuaded to; the most clear thesis statements I see above are exactly about artifacts and systems. Good problems are likely to be scarce because of our choice of artifacts and systems.

1h

@drScott@tao

This one:

but at the cost of sustaining the ecosystem for the next wave of progress

I don't believe there is any cost incurred when AI companies solve open math problems. If the goal is to produce insights and tools, existence of an inscrutable proof takes away nothing, and only adds.

1h

@nilesh@tao I understand the argument for that claim as follows:

1. If a mathematician is scooped on a problem, it is hard for them to disseminate their insights and tools.

2. AI can, with sufficient investment, convert a tractable problem into a solution (and thus a scoop).

3. Mathematicians can (only?) avoid being scooped by being secretive about their work.

4. Good open problems are hard to make without an open mathematics community.

(2-3) have just been given a major example, and seem quite plausible. (1) has a long history in the field, no? Plenty of careers ended or interrupted, and several famous and messy fights over priority. I think Tao argues for (4) persuasively.

29m

@drScott@tao

I need to ponder more on this. I was reasoning under the scenario where most people have equal access to frontier AI.

So, using AI becomes akin to using a calculator or a personal computer. But currently, it's definitely unequal (OpenAI reportedly spent more than $1m for Navier-Stokes), which is where "scooping" becomes possible, and might encourage not sharing promising research directions in the first place.

2h

@tao I would like to add that the current situation is also seriously damaging the training and development of mathematicians, particularly early-career researchers who have not yet secured tenure. Publication inflation makes it difficult for them to concentrate on their research. They feel anxious and often seriously question whether their work has any real value. In particular, the uncertainty this creates about how the mathematical community and universities will evaluate research in the future leaves them with little hope for their careers. If the current situation drives young mathematicians out of the field in large numbers, creating a generational gap, I believe this will be even more detrimental to the development of mathematics.

1h

@tao yeah this is the most depressing part to me

Sulfide_Sleuth@PhilGopon@mountains.social2h

@tao

I remember having to do a literature data compilation for part of my PhD, that took the better part of 4 months.

Today I recreated the same survey in less than 20 minutes with Claude, and I don't really think that is a good thing.

Data compilations used to force you to read and think about the other work, and in the end you had something you understood and could nicely write up to help others understand.

The speed of publishing is outpacing the ability of ppl to read it. It was a problem before AI and now it is just out of control.

AI to create papers that only AI will actually read, is where we are at now.

All this at the cost of the exponential increase in energy and water resources, where is the benefit?

spicy_ricecaker@spicyricecaker@fosstodon.org2h

@tao Reminds me of Janko Meteldon's post on The Cognitive Dark Forest...

3h

@tao#ai still FAILS at tasks where there is not sufficient data, such as #tetris in #bash

1h

@wordmark@tao The 'bash tetris' example is arguing in favor of mathematics going dark? I think that would be sad.

I also suspect that self-play would make tetris in the terminal easy for AI, but that's not so crucial.

1h

@wordmark@mas.to@tao@mathstodon.xyz i was able to one-shot tetris in bash with gemini 3.1 pro (a 7 month old model) https://bin.disroot.org/?750ff7153320b1d4#521SqCvhdp72uE4qXS9MdALX4ntRUdjoiE9xoLHLe4Ra

admittedly, it lacks holding and queue but that probably can be achieved in another prompt

not saying llms are perfect but you might want to reevaluate their power

Disroot Bin - Encrypted pastebin by PrivateBinDisroot Bin - Encrypted pastebin by PrivateBinVisit this link to see the note. Giving the URL to anyone allows them to access the note, too.

poetaster@poetaster@mastodon.gamedev.place3h

@tao ah, who opened Pandora's box?

arsonatdennys@arsonatdennys@kolektiva.social1h

@tao would deeply appreciate a bit of elaboration on “to the point where one can no longer discern its geometry to the extent that promising questions can be extracted within the range of applicability of the tool. Often this is counteracted by the ability of such a tool to enlarge the radius …”

The metaphor is of concentric circles where the interior of inner circle’s geometry has been flattened by this new tool but area b/w the outer circle and inner circle still has sufficient ‘curvature’ to identify promising questions?

and here, is promising in more than (just) understanding the difficulty boundaries of the tool? E.g. difficulty topography is a compass to other qualities of interest.

18m

@tao

I really love how you think about the difficulty landscape of a field in such metamathematical ways!

"Every new advance in mathematics, whether it comes from technique, technology, or infrastructure (such as access to libraries of past literature) reduces the difficulty of solving problems.

This is generally a good thing; but it comes at the cost of flattening out the difficulty landscape of a field, to the point where one can no longer discern its geometry to the extent that promising questions can be extracted within the range of applicability of the tool.

Often this effect is counteracted by the ability of such a tool to enlarge the radius of the sphere of results one can plausibly reach, creating new boundaries to fruitfully explore."