OpenAI Math Repo Explained: 722 AI Papers, Lean Proofs, Backlash

By Beau Johnson··10 min read

Quick answer

On October 6, OpenAI posted 722 AI-written math papers to its openai/math GitHub repo. Only about 42% of the top-line results are formalized in Lean, and Lean checks the proof against the formal statement, not the paper. Builders using coding agents run into the same verification gap.

In this video I break down OpenAI's release of 722 AI-written math papers: what it published, the one result experts are excited about, and why a lot of mathematicians are upset. I also explain how the Lean proof checker works, the trap built into it, and why anyone running coding agents already has this same problem. You don't need a math background to follow along.

What I cover in this video

  • 0:00 722 papers, and three pulled over one minus sign
  • 0:45 What OpenAI actually dropped on GitHub
  • 3:00 The headline result: the quasi-Riemann hypothesis
  • 4:34 The backlash from mathematicians, and the other side
  • 6:50 How Lean checks a proof in four steps
  • 9:53 What the 42% figure really covers
  • 11:24 My take for builders running coding agents
  • 12:56 Proof first, understanding second

What OpenAI actually dropped

On October 6, a new repo called openai/math showed up on GitHub. The first commit was signed "anonymous," and the repo now has over 17,000 stars. OpenAI says its models had maxed out (their word is "saturated") their existing math tests, so it started testing them on open research problems. The model got about 4,000 problems, and each result used the equivalent of about three hours of ChatGPT Pro thinking compute on average. OpenAI grouped the output into families and kept what it considered significant. Scott Aaronson's rough math puts that at about 1 in 10.

The result was 722 papers in 372 families. The README calls the model "an unnamed internal OpenAI model," and OpenAI says it's working to release it responsibly. So you can't use the model. You only get the papers. Some of the claims are huge: a counterexample to Hadwiger's conjecture, Hilbert's 10th problem over the rational numbers, and, added October 26, a paper claiming the exponential time hypothesis is false. A group of computer scientists led by Russell Impagliazzo, Ramamohan Paturi and Ryan Williams will examine that last one. Every one of these is still a claim.

The quasi-Riemann result

The Riemann hypothesis has been open for over 160 years. It's about where the zeta function equals zero, and those zeros are tied to how the prime numbers thin out. We already know the important zeros sit in a strip between 0 and 1. The quasi-Riemann hypothesis asks you to prove they all sit in a narrower strip. That's what OpenAI's paper claims: no zeros with a real part past 7/8. It's a weaker version, not the Riemann hypothesis itself, but it's still a big deal.

There are two papers. A 199-page paper gets to 7/8, and a 49-page paper gets to 11/12. The second one is the only paper out of all 722 that a human edited for readability. According to Quanta, experts largely agree the result is correct. Peter Sarnak, though, won't call it a theorem until humans have checked and understood every detail. The odd part is that the experts mostly studied the short, human-edited paper, which has no Lean proof. The Lean proof is for the long one. That's two different checks on two different papers.

The backlash, and the other side

Hours before the repo went up, Terence Tao posted a thread on Mastodon. It doesn't name OpenAI and it isn't anti-AI. His concern is the way this is being done. In his words, solutions to open problems are "being harvested at large scale in an unsustainable fashion." Then the Association for Human Mathematics called the release "not a demonstration of scholarship, but a demonstration of power" and urged mathematicians to stop working with OpenAI. Quanta described the group as over 800 mathematicians. One myth to clear up: Tao reposted that statement on his blog, but he didn't sign it and he isn't a member.

On the other side, Scott Aaronson's post "The Mathocalypse" is mostly amazed, though he also says the field should hold AI companies' feet to the fire. Jeremy Avigad told Quanta that releasing the papers is generally a good idea. Both sides have a point, and underneath it they're arguing about the same question: who is going to check all of this?

How Lean checks a proof, and the trap inside it

Lean is a programming language that lets a computer check mathematical proofs. I walk through it in four steps:

  1. The statement. You write the claim as code, built on Mathlib, a shared library of math people have already formalized. In the challenge file the proof is left as sorry, which is the blank the proof has to fill in.
  2. The proof. For the zeta result, it's about 2,900 Lean files and nearly half a million lines by our count. OpenAI lists the method as "agent," so nobody reads that by hand.
  3. The kernel. This is a small, trusted type checker that checks every step against the rules. It checks the Lean proof. It doesn't check the 199-page paper.
  4. The escape hatches. sorry can be used to prove anything, so Lean warns you when a finished proof still contains one. Extra axioms work the same way. OpenAI's repo includes a tool called Comparator that only allows three standard axioms and confirms the solution proves the same statement as the challenge.

Here's the trap. Lean checks that the proof matches the statement. It can't check that the statement matches the paper. Johan Commelin, who runs the Mathlib Initiative, told Quanta that this takes human experts reading the Lean code. He also said some formalizations appear to prove weaker statements than the papers claim, though Quanta notes he hadn't confirmed that firsthand. Lean is a strict grader for whatever question you wrote down. If you write an easier question, it will mark the easy answer correct.

The 42% problem and the minus sign

On October 6, OpenAI told Quanta that about half the results were formally verified in Lean. On October 7, the repo gave a real number: 300 out of 719 top-line results, or about 42%. There's been no newer figure, even after more formalizations were added October 26. OpenAI doesn't define "top-line results," and when we counted from the repo's own files three different ways, none of them came out to 300. I'm not saying 42% is wrong. You just can't check it from the outside. OpenAI's own catalog also marks the review status as "unchecked," which means there's no reported review of whether the statements match the papers.

The sign error happened in the unformalized part. A paper on Weyl classes gave each reverse trace a sign of +1 when it should have been −1. A count that needed to come out to zero came out to −2m instead. Two other papers relied on the same construction, so all three were withdrawn. None of them had a Lean formalization listed. The notice says the withdrawal is about the proof, not a claim that the statement is false. That same week, OpenAI also revised 14 other manuscripts.

My take for builders

This is the same problem anyone running coding agents already has. Once AI writes more than humans can review, verification becomes the job. Every piece of Lean has a match in a Claude Code workflow:

  • The formal statement is your spec, your acceptance test. Write it before the agent starts, in words you can actually read.
  • The kernel is your CI. It runs every time and can't be talked out of a fail.
  • sorry is the skipped test, the TODO stub, the "handle this later" comment. Search for those before you merge.
  • The weaker statement is a test that only checks your function returns something, not the right thing. In my experience, agents love writing those.
  • Comparator is a locked test. The agent can read it but can't edit it. That's also why on my model benches, two of the four tests use hidden tests the model never sees.
  • Use a reviewer that isn't the author. That can be another sub-agent, another model, or you, and it should read the spec, not only the diff. Every one of my video scripts goes through a fact-check agent that didn't write it.

If you're building with AI agents, keep in mind that your tests can all pass and still be the wrong tests.

Key takeaways

  • OpenAI posted 722 AI-written papers from an unnamed model. Three were pulled within about a day over a single sign error.
  • The quasi-Riemann result has experts impressed, but the Lean proof and the human-edited paper experts studied are two different papers.
  • Lean checks that a proof matches its formal statement. It can't tell you whether that statement matches the paper.
  • OpenAI says about 42% of top-line results are formalized, but "top-line" isn't defined and the number can't be checked from the outside.
  • For builders: write the spec first, lock your tests, search for stubs, and have someone other than the author review the work.

FAQ

What is the openai/math GitHub repo?

It's a repo OpenAI published on October 6 with 722 math papers in 372 families of results, written by an unnamed internal model. OpenAI posed the model about 4,000 problems and kept the results it considered significant.

Did OpenAI's AI prove the Riemann hypothesis?

No. It claims a weaker version called the quasi-Riemann hypothesis: no zeta zeros with a real part past 7/8, in a 199-page paper with a Lean proof. A second 49-page paper gets to 11/12 and has no Lean proof yet.

Why were three of OpenAI's math papers withdrawn?

One paper used a sign of +1 where it should have used −1, so a count that needed to be zero came out to −2m. Two other papers relied on the same construction. None of the three had a Lean formalization listed, and the notices say the statements aren't shown to be false.

Does a Lean proof mean an AI math paper is correct?

Not entirely. Lean's kernel checks that the proof matches the formal statement, but it can't check that the statement matches what the paper claims. That still takes human experts reading the Lean code.

Chapters

  1. 0:00722 papers and one minus sign
  2. 1:07What OpenAI actually dropped
  3. 2:35Join Shipping Skool
  4. 2:55The headline result: quasi-Riemann
  5. 4:55The backlash (and the other side)
  6. 7:13How Lean checks a proof (and the trap)
  7. 9:50The 42% problem
  8. 11:39My take for builders
  9. 13:26Proof first, understanding second

Transcript

Read the full transcript

0:00722 math papers posted to GitHub in one afternoon by an AI model OpenAI hasn't even named. And within about a day, three of them were pulled because of a minus sign, a plus one that should have been a minus one. On October 6th, OpenAI put up one repo. The model had been posed about 4,000 problems. 722 papers came out in 372 families of results. And OpenAI says about 42% of its topline results are formalized in lean. So who checks the rest? Because a group Quanta described as over 800 mathematicians called on the field to stop working with Open AI. And in this video, I'm going to be breaking

0:45down what OpenAI actually dropped, the one result experts are genuinely excited about, why so many mathematicians are furious, how a computer checks a proof, and the trap hiding inside it. what that 42% really means and why this is the exact same problem anyone running coding agents already has. Let's jump right into it. So, what actually dropped? On October 6th, a brand new repo showed up on GitHub, OpenAI/Math. The very first commit was signed anonymous. It's now sitting at over 17,000 stars. So, here's how OpenAI says it was made. They test their models on open research problems and they expanded that after the models maxed out their

1:30existing math tests. Their word is saturated. The model was posed about 4,000 problems. Each result used on average the equivalent of about 3 hours of chat GPT proinking. Then they grouped what came out into families and kept what they felt was significant enough. Scott Aronson's rough math that's about 1 in 10. And the model, the readme calls it an unnamed internal OpenAI model. OpenAI says it's working to responsibly release it. So you can't use it. You just get the papers. And the claims are wild. A counter example to Hadwigger's conjecture. Hilbert's 10th problem over the rational numbers. And on October 26th, they added one called the exponential time hypothesis is false. A group of computer scientists led by

2:15Russell Impagliato, Rammoan Patoui, and Ryan Williams will examine that one. And remember, every single one of these is a claim. On launch day, Quantum magazine put it this way. For those inclined to anthropomorphize computer software, OpenAI's anonymous new model today became one of the most prolific mathematicians in history. Real quick, every video I make is built in code with Claude Code. Inside Shipping School Premium, I show you how step by step to turn your product into videos that get views, plus new courses every day, 500 plus hours of replays, live calls every week, and a Q&A with me every Saturday. Join free link below. So, let's start with the one people are actually excited about, the Remon hypothesis. It's been

3:00open for over 160 years, and it's arguably the biggest open question in math. It's about a function called zeta and the places where it equals zero. Reman showed that how the prime numbers thin out is tied to those zeros. And we already know the important zeros live in a strip between 0 and 1. So the quasi remon hypothesis says prove they all live in a narrower strip. And that's what OpenAI's paper claims. No zeros past 7/8. It's a weaker version. It is not the Remon hypothesis, but it's a big deal. There are actually two papers on it. The first one is 199 pages and gets you to 7/8. The second one is 49 pages and gets you to 1112ths, a little weaker, way shorter. And that second one

3:47is the only paper out of all 722 that a human edited for readability. Who? OpenI told quant the paper is the result of work from everyone at OpenAI. Come on now. So this is where the experts get impressed. Jared Litman at Stanford told Quant, "It's the most significant AI for math achievement so far by quite a margin." Maxim Radzowil at NYU told Quanta, "The implications are staggering." He also said, "Both papers are horribly written." His words, not mine. So, Quanta reports, "The experts largely agree the result is correct. But Peter Sarnac won't be satisfied it's a theorem until humans check and understand all the details." And here's the funny part. The experts mostly studied the short human edited paper and

4:34that one has no lean proof. The lean proof is for the long one. Two different checks on two different papers. And Levent Alpogue, a mathematician at Anthropic, OpenAI's rival, posted on day one, it's obviously the most significant moment in mathematical history. A mathematician at Anthropic said that about OpenAI. So why are so many mathematicians this upset? Start with Fields medalist Terrence Tao. Hours before the repo even went up, Tao posted a thread on Mastadon. It doesn't name open AI and it isn't anti- AI. It's about how this is being done. His point is once a problem is solved, it can't be unsolved. And the old way, what he calls math 1.0, left whole fields more

5:20fertile. So in his words, solutions to open problems are now being harvested at large scale in an unsustainable fashion. And to the New York Times, he said, I'm on a war footing. Then the Association for Human Mathematics put out a statement. Mathematicians did not ask for this work to be done. And this line, releasing over 700 files at once is not a demonstration of scholarship, but a demonstration of power. They urged mathematicians to stop working with open AI and the press called it a boycott. Quanta described the group as over 800 mathematicians and its member list includes Fields medalist Peter Schultz. And quick myth to clear up, Tao reposted that statement on his blog. He didn't sign it and he's not a member. Mihales

6:05DeFermos at Princeton told Quanta that in their current form at least, the papers look far from constituting in of themselves a proof, let alone a proof that a person could verify. Quanta summed it up as essentially not even wrong. Michael Harris at Colombia told Nature, "I will not be reading these papers." And Ursula Martin at Oxford told Nature, "It's tossing the community a messy first draft and expecting us to clean it up for them." Now, the other side, Scott Aronson wrote a blog post called the math aocalypse, and he's mostly amazed. He called it surely one of the biggest days in mathematical history. Though he also says the math community should hold the AI company's feet to the fire to behave better. Jeremy Avagad told Quanta it's against the spirit of mathematics to keep

6:50secrets so releasing it is generally a good idea. And Gnome Brown at OpenAI says a lot of the researchers behind this are math PhDs or former math professors and if anything they care too much about the future of mathematics. So both sides have a point. But listen to what they're actually fighting about. Who is going to check all of this? And that's where leen comes in. So what is lean? OpenAI calls it a programming language that allows mathematical proofs to be checked by a computer. You write the math as code and the computer checks it. And don't get lost in the sauce here. It's four steps. Step one, the statement. You write the claim as code. Here's the real one from OpenAI's repo for that Zeta result. In plain English,

7:35it says if the real part of S is more than 7/8, then zeta of S isn't zero. It's built on Mathlib, which is a shared library of math people have already formalized. And see that word at the very end, sorry, that's on purpose. This is the challenge file. So sorry is the blank the proof has to fill in. So step two, the proof that lives in separate files for the Zeta result. By our count, it's about 2900 lean files, nearly half a million lines. And OpenAI lists its lean method as agent, so you're not reading that by hand. Step three, the kernel. Lean's docs call it the trusted kernel, and it's a small type checker. It takes the proof and checks every step against the rules. It doesn't care who wrote it or how ugly it is. So, all you

8:22have to trust is one small checker. But notice, it checks the lean proof. It does not check the 199 page paper. Step four, the escape hatches. Lean's docs say sorry can be used to prove anything. So if a finished proof still has one, lean warns you. Declaration uses sorry. Extra axioms are the same idea. Assumptions you just declare true. So OpenAI's repo ships a check with a tool called comparator. It takes the challenge file and the solution file and it only allows three standard axioms. If it passes, the proof is guaranteed to prove the same statement as the challenge, use no extra axioms, and be accepted by the lean colonel. So, here's the catch. Lean checks that the proof matches the statement. It can't check

9:07that the statement matches the paper. Johan Klan, who runs the Math Lib initiative, told Quanta, "That's not something that can be established automatically. It needs human experts to read the lean code." And he said some of the formalizations appear to prove weaker statements than what's claimed in the paper or only formalize parts of it. Quant notes he hadn't confirmed that firsthand, but that's the trap. Lean is a really strict greater for the question you wrote. Write an easier question and it'll happily mark the easy answer, right? And to be fair, OpenAI documents some of this. For the Zeta result, their own scope note says the paper's later applications are not included. That's documented. It's not an error, but it shows you the gap. So now that 42%. On

9:53October 6th, Quanta reported that according to OpenAI, about half of the results had been formally verified in lean. The readme that day just said many, but not all. Then on October 7th, the repo gave a real number. 300 out of 719 topline results formalized, about 42%. And there's no newer number. even after they added more formalizations on October 26th. Here's something to be wary about. OpenAI doesn't define topline results. We tried counting from the repo's own files three different ways and none of them came out to 300. So depending on how you count, you get a different number. I'm not calling 42% wrong. You just can't check it from the outside. So OpenAI's own catalog marks

10:38the review status as unchecked. That doesn't mean the proofs weren't machine checked. It means there's no reported review of whether the statements match the papers, which is exactly Klan's trap. Quanta also found a handful of the lean statements had technical issues. Easy fixes, but in their words, the mere existence of a lean proof is not the final word, and the rest isn't formalized yet by OpenAI's own count, which is exactly where the minus sign was. One paper on while classes gave each reverse trace a sign of + one. It should have been minus one. So with m of them, a count that was supposed to come out to zero came out to minus2 m and the argument needed that zero. Two other papers relied on the same construction. So all three got pulled. None of the

11:24three had a lean formalization listed. And OpenAI's notice says the withdrawal is about the proof. It does not assert the statement itself is false. That same week, they also revised 14 other manuscripts with proof repairs and corrected statements. So here's my take and this is my opinion. This is the exact problem anyone running coding agents already has. When AI writes more than humans can review, verification becomes the job. A viewer under the 2-minute papers video said it best. AI submits 700 pull requests. Mathematicians, nah, I'm not merging this into Maine. And the way I think about it, every piece of lean maps right onto cloud code. The formal statement is your spec, your acceptance test. Write it before the agent starts in words you

12:10can actually read. The kernel is your CI. It runs every single time and it can't be talked out of a fail. Sorry is the skipped test, the to-do stub, the handle this later comment. Search for it before you merge. The weaker statement is the test that checks your function returns something instead of the right thing. And in my experience, agents love writing those. So comparator is a locked test. The agent can read the challenge, but it can't edit it. That's also why on my model benches, two of the four tests are scored by hidden tests the model never sees. The model never grades itself. And the big one, a reviewer that isn't the author. The colonel checks the lean proofs that exist. What's missing is someone independent checking that the statements say what the papers say. Same

12:56thing in a repo. Your tests can all pass and still be the wrong tests. So use a separate reviewer, another sub agent, another model, or you. and have it read the spec, not only the diff. Like I said, I build these videos with claude code, and every script goes through a fact check agent that didn't write it. Tao said something to the Times that stuck with me. There is no author that I can have a real conversation with about the results. Same with your agent. It's not accountable for what gets merged. You are. So, proofs used to be how humans learned something new. You fought with a problem and the understanding came with the proof. Now the proof can show up first. Quanta reports Peter Sarnac is turning a course he's teaching this semester into a study of the Zeta

13:42paper. Radzawil can now sketch the argument in half a page. And the Simons Foundation, Anthropic, and Open AI are putting $15 million in combined funding and resources into helping mathematicians navigate all of this. So the understanding is coming. It's just coming second. And for builders, I think that order matters. build the checker first. So, real question, would you merge 700 pull requests you couldn't read? What's your rule for agent code? Let me know in the comments and give this one a like if you're running agents, too. Have a blessed day.

Ready to start building with AI?

Join the Shipping Skool community free. Paid members get live bootcamp calls every week and weekly-updated courses.

Join Shipping Skool

Keep reading