Open Weight Thoughts
All articles

· 8 min read

Can AI Discover New Mathematics Instead of Just Proving Things Humans Already Suspect?

By R. Yamamoto

  • guides

Yes—AI can contribute genuinely new mathematics, not merely fill in proofs for conjectures humans already wrote down. But it works best when the problem gives the system a tight feedback loop: generate a candidate, evaluate it mechanically, preserve the useful mutations, then formally verify and interpret the result.

That distinction matters. A language model producing a plausible theorem in a chat window is not mathematical discovery. A system finding a previously unknown construction that improves a known bound, survives exact checking, and teaches researchers something reusable is much closer to the real thing.

Discovery is a different job from proving

It’s tempting to frame mathematics as one task: start with assumptions, end with a proof. In practice, researchers do at least three different kinds of work. They notice patterns, formulate a worthwhile statement, and then prove it. Current AI is strongest at the third step when it can use a proof assistant, symbolic solver, or narrow formal language. The first two are where “new mathematics” lives—and where evaluation is much harder.

A proof has a crisp success condition. Lean, Isabelle, Coq, or a domain-specific verifier can say whether each inference checks. A conjecture has no equivalent unit test. It can be true but trivial, false in an uninteresting way, secretly already known, or too weak to matter. Novelty and importance require comparison against literature, nearby examples, and a researcher’s sense of what opens a path forward.

So the practical question isn’t “can an LLM be creative?” It’s “can we turn mathematical creativity into a search problem with strong enough feedback that good ideas outcompete nonsense?” In a few domains, the answer is already yes.

What the credible successes actually show

FunSearch is a useful model for software engineers because it is not a chatbot pretending to reason. It uses an LLM to propose small programs, evaluates those programs with a problem-specific scorer, and feeds high-performing candidates back into an evolutionary search loop. In extremal combinatorics, the system found constructions for cap-set problems that improved known lower bounds in some dimensions. The important ingredient was not eloquent output; it was that every candidate program had an executable fitness function.

AlphaGeometry illustrates a neighboring pattern. It combines a learned component that proposes useful auxiliary constructions with a symbolic deduction engine that checks consequences. It solved 25 of 30 olympiad-level geometry problems in the reported benchmark, and its generated constructions could lead to proofs that are not obvious from the problem statement. That is impressive reasoning and search, but it is still substantially different from independently choosing a new research direction and judging why it matters.

There are also systems aimed directly at conjecturing. Earlier machine-learning work in topology and representation theory used models to expose patterns in data, then had mathematicians turn those observations into conjectures and proofs. This is closer to the historical role of computation in math: not an oracle that replaces understanding, but an instrument that makes a structure visible.

The recipe: generator, evaluator, verifier, interpreter

If you want to reason about whether an AI math project can discover anything, ignore the model name at first. Inspect the loop around it. A serious system separates proposing ideas from testing and validating them.

  1. Define a constrained representation. Candidates might be programs that construct combinatorial objects, expressions in a grammar, graph transformations, or formal proof steps. The representation must be searchable and cheap enough to manipulate at scale.
  2. Build an evaluator before building prompts. For a cap-set construction, calculate its size and test its constraints. For an algorithm, measure its competitive ratio or runtime on exact instances. The evaluator supplies the selection pressure a generic LLM lacks.
  3. Use the model as a proposal distribution, not as the final authority. Sample many candidates, mutate promising ones, retain diversity, and archive every result with its score and provenance.
  4. Add independent verification. Use a proof assistant when possible; otherwise use exact arithmetic, multiple implementations, adversarial test generation, and human review. An LLM checking its own argument is not independence.
  5. Reserve human time for interpretation. Ask which surviving results are novel, what invariant explains them, whether a pattern generalizes, and what theorem would make the result legible to another mathematician.

This should sound familiar. It is close to property-based testing, fuzzing, compiler optimization, and evolutionary program synthesis. The LLM is good at proposing structured edits that are more semantically useful than random noise. The evaluator decides whether those edits deserve another generation.

Why general-purpose chat is still unreliable for this

A frontier model can often produce a proof-shaped answer with a hidden gap. Mathematics is especially hostile to this failure mode because one invalid quantifier change, unproven regularity assumption, or edge case can invalidate pages of otherwise sensible work. The answer may even contain real-looking citations, established terminology, and locally correct algebra. None of that makes the central claim true.

Training data also complicates claims of discovery. If a model restates a theorem from a paper, recombines a familiar lemma, or solves a public benchmark, that may be useful but it is not evidence of new mathematics. A convincing discovery workflow needs to document what the system had access to, what was generated, how candidates were selected, and how the final result was checked. Reproducibility is not paperwork here; it is the boundary between a promising anecdote and a result.

Even a verified theorem is not automatically valuable. Search systems naturally optimize what you can measure, so they can produce a mountain of correct but ugly special cases. Mathematicians often want compression: a definition, invariant, or proof method that explains many results at once. AI can help find the mountain. It is not yet consistently good at deciding which trail changes the map.

A concrete project you can build

Don’t start by asking a model to solve the Riemann hypothesis. Pick a domain where correctness and quality are executable. For example, search for small graph constructions that maximize a target property while avoiding forbidden subgraphs, or search for short programs that generate integer sequences with specified constraints.

Write a candidate format and a deterministic evaluator. Have an LLM generate or mutate candidates, but insist that it outputs only the constrained format—say Python functions, a tiny DSL, or Lean declarations. Run candidates in a sandbox, score them, store the top results plus diverse outliers, and periodically ask the model to explain recurring structures rather than merely improve the scalar score.

Then separate two dashboards. One tracks optimization: best score, pass rate, novelty relative to your archive, and compute spent. The other tracks mathematics: which patterns recur, which conjectures hold on exhaustive small cases, and which statements you can formalize. If the first dashboard improves while the second remains empty, you have built a clever search benchmark, not a discovery system.

That is the near-term answer. AI can discover new mathematics when we give it a world where hypotheses can be generated, competed, falsified, and checked. The bottleneck is increasingly not whether a model can emit an interesting idea; it is whether engineers and mathematicians can build feedback loops that distinguish an interesting idea from a very convincing pile of tokens.

Sources & citations

  1. [1]Nature: Mathematical discoveries from program search with large language models
  2. [2]Nature: Solving olympiad geometry without human demonstrations
  3. [3]Nature: Advancing mathematics by guiding human intuition with AI