First Proof Benchmark Exposes Wide Gap Between Proprietary and Open AI Math Skills

Key points

• First Proof’s third benchmark shows proprietary models vastly outperforming open-weight models in research mathematics.
• ProofCouncil, a hybrid system using OpenAI’s GPT 6 Astra, solved nine of ten peer-reviewed math problems.
• The leading open-weight model, Kimi K3, solved zero problems, confirming a large gap in capability.
• Mathematicians are increasingly concerned about data privacy when using proprietary AI, driving demand for reliable open alternatives.
• The benchmark follows OpenAI’s controversial 'slop drop' and Google Deepmind’s peer-reviewed AlphaProof Nexus results.

Background

In early 2026, AI models began achieving notable success in mathematics, initially on simpler problems. By October 2026, the field saw rapid escalation: OpenAI released over 700 AI-generated proofs in a single 'slop drop,' claiming to solve 372 open problems, which sparked backlash and retraction of some work. Simultaneously, Google Deepmind’s AlphaProof Nexus solved dozens of open problems, including two unsolved for 56 years, and underwent peer review. These events prompted the creation of the First Proof project in February 2026 by top mathematicians to establish a rigorous, independent benchmark for evaluating AI mathematical reasoning, moving beyond imperfect and inconsistent prior benchmarks.

How outlets are covering it

Scientific American emphasizes the stark divide between proprietary and open-weight models, quoting Stanford mathematician Mohammed Abouzaid on the 'very large' gap in research mathematics and Harvard’s Lauren Williams on the failure of open models. The Decoder highlights Google Deepmind’s AlphaProof Nexus as a peer-reviewed success, contrasting it with OpenAI’s unverified mass release. Wired frames the broader trend of AI expanding beyond math into other scientific disciplines, while the archive notes the community’s anxiety over verification and the potential displacement of human researchers. The sources agree on the rapid progress of frontier models but differ in emphasis: Scientific American focuses on the accessibility gap, while The Decoder spotlights specific peer-reviewed achievements by Deepmind.

Why it matters

The First Proof results demonstrate that AI is now a viable tool for high-level mathematical research, but only for those with access to proprietary, subscription-based models. This creates a significant inequity in scientific advancement, as open-weight models remain inadequate for complex problems. The privacy concerns surrounding proprietary AI, highlighted by mathematicians wary of data leakage, may drive further investment in open-source alternatives. The benchmark also underscores the need for rigorous, independent evaluation of AI capabilities, as unverified claims from companies like OpenAI have led to retracted work and academic controversy. Ultimately, the gap between frontier and open models could shape the future of AI-driven research, influencing who benefits from these technological advances.

What to watch

The First Proof team expects to continue testing models as capabilities evolve, with potential future benchmarks focusing on tasks beyond proving well-defined statements. Mathematicians are likely to push for improved open-weight models to address privacy and accessibility concerns. The debate over AI’s role in mathematics may intensify, with calls for stricter verification standards and ethical guidelines for AI-generated proofs. Google Deepmind and other labs may respond to the benchmark by enhancing their models’ transparency or performance. The broader scientific community will monitor whether AI can extend its success to other disciplines, as suggested by Wired, while addressing the equity and verification challenges exposed by the First Proof results.