GPT-6.2 Astra vs Sol vs Claude Fable 5.2: 4 Coding Tests and Costs

By Beau Johnson··8 min read

Quick answer

I ran GPT-6.2 Astra and Sol in Codex against Claude Fable 5.2 and Sonnet 5.5 in Claude Code on four tests. Sol won the bug hunt for 89 cents and cost $2.43 across all four tests, versus $9.14 for Astra and $35.88 for Fable (API equivalent).

OpenAI released GPT-6.2 Astra and GPT-6.2 Sol on November 17, and the early reviews disagreed with each other. In this video I run both as coding agents in Codex against Claude Fable 5.2 and Sonnet 5.5 in Claude Code on four tests, with a stopwatch on every run. If you build with coding agents and want to know which model is worth paying for, this one is for you.

What I cover in this video

  • 0:46 What Astra and Sol are: price, context window, and the new Ultrafast mode
  • 2:18 The independent check from Artificial Analysis
  • 4:36 What people built on day one, and the rough reviews
  • 7:41 Test 1: a Star Fox game
  • 9:16 Test 2: a real client job with 12 hidden tests
  • 12:17 Test 3: the pelican on a bicycle, in 3D
  • 13:49 Test 4: the bug hunt with 16 hidden bugs
  • 16:51 Total cost and who should switch

What GPT-6.2 Astra and Sol are

Astra is OpenAI's frontier model, built for your hardest projects. Sol is the everyday one. Both have a 1 million token context window, and both are in Codex for Plus, Pro, Enterprise, and Edu, plus the API.

Astra is $8 in and $40 out per million tokens, the exact same price as Claude Fable 5.2. Sol is $2 in and $10 out, the exact same as Sonnet 5.5. So the matchups pick themselves. There's also a new Ultrafast mode, up to four and a half times the normal speed. You get it on both models with the $500 Pro plan and on Sol only with the $200 Pro plan.

OpenAI says Sol lands within half a point of Astra on a coding test for about a fifth of the cost. Artificial Analysis ran both on launch day. On their Intelligence Index, Fable 5.2 is first at 65, Astra is 62, Gemini 4 Carbon is 60, Opus 5.5 is 59, and Sol is 57. The cost to run one task on that index is $26.54 for Fable, $4.00 for Astra, and $1.01 for Sol. Astra and Fable have the same sticker price, but Astra writes far less: about 66 million output tokens to run the whole index against Fable's 275 million.

The coding picture was less friendly. On Artificial Analysis's coding agent index, Astra was flat with GPT-6 Astra and Sol dropped three points. Matt Shumer said, "They were essentially unusable." Dan Shipper at Every said that in real coding tasks Astra seemed to stop too early. I wanted to see it for myself.

How I set up the four tests

Astra and Sol ran in Codex on a ChatGPT login. Fable 5.2 and Sonnet 5.5 ran in Claude Code on my Claude subscription. Every model got the same prompt, one shot, no follow ups, high effort, web search off, and a fresh empty folder. Each result is one finished run.

The costs I show are API equivalent, meaning what the same run would cost on the API. Nothing was billed per run.

One note for the record. Our ChatGPT Plus plan hit its Codex usage limit partway through Sol's client job. I upgraded the plan, and Sol's client job and pelican ran again on the same settings.

The four tests and what happened

Test 1: a Star Fox game

An on rails space shooter in Three.js. All four built a playable game. Sol finished first at 11.8 minutes for $0.59, then Sonnet at 15.0 minutes for $1.66, Astra at 21.3 minutes for $2.37, and Fable at 24.8 minutes for $8.20. To me, Astra's looked the most finished and Fable's had the most game in it, with three canyons, a space section, and music and sound effects made in code.

Test 2: a real client job

A small landlord wants a maintenance request portal. The email thread is messy on purpose: a requirement changes midway, and the client asks for emergency texts that a single web page can't send. All four scored 12 of 12 on the hidden tests and all four caught the changed requirement. Sonnet took 3.1 minutes for $0.50, Sol 8.1 minutes for $0.39, Astra 9.4 minutes for $1.10, and Fable 11.4 minutes for $4.42.

Two outside models judged the reply email to the client. Fable and Sonnet got 10 and 10, Astra 9 and 10, Sol 8 and 9. Astra handled the texting trap the way I'd want a real developer to:

Emergency texts need a separate service. This file shows a banner in manager while any emergency is open. It cannot text Marcus. Please keep using your existing emergency phone procedure.

Test 3: the pelican on a bicycle, in 3D

Sol was quickest at 13.2 minutes for $0.57. Sonnet took 14.3 minutes for $1.74, Fable 32.0 minutes for $11.23, and Astra 34.9 minutes for $2.38. Both OpenAI models gave their scene the same designed look, with a title and a frame around everything. Astra has a house style, and you can see it here.

Test 4: the bug hunt

A Node invoicing backend, about 3,000 lines across 42 files, with 16 hidden bugs and 11 decoys. False reports cost points, and two outside models judged the explanations. Scores are out of 32.

  • Sol: 32 and 31, fixed 16 of 16, 16.8 minutes, $0.89
  • Fable 5.2: 30 and 30, fixed 15 of 16, 28.7 minutes, $12.04
  • Astra: 30 and 29, fixed 16 of 16 with two or three false reports, 29.6 minutes, $3.29
  • Sonnet 5.5: 28 and 28, fixed 14 of 16, 4.1 minutes, $0.96

The cheap one won. Nobody claimed a fix that didn't work. Astra went the other way and flagged too much.

Which one I'd pay for

Across all four tests, Sol came to $2.43, Sonnet $4.87, Astra $9.14, and Fable $35.88. Astra and Fable took about the same total time, but Fable wrote about five times as many output tokens, so it cost about four times as much.

If you're on ChatGPT Plus, start with Sol. It also stretches that five hour usage limit further, which matters because we hit it. Use Astra when the work has to look good, and tell it the look you want. Fable 5.2 still did the most careful work, with zero false reports. If you're on Claude Max, where Fable is part of the plan, the API cost matters a lot less. Sonnet was the quickest overall at about 37 minutes for all four tests.

In my resale business, small data jobs like tracking shipments are Sol work. Anything a customer sees, I'd hand to Astra or Fable. For more on the two tools themselves, see Claude Code vs Codex.

Key takeaways

  • Sol won the bug hunt and cost about a quarter of what Astra did across all four tests.
  • Astra and Fable 5.2 have the same price per token, but Fable writes far more, so its API cost was about four times higher.
  • Astra's 3D builds looked the most designed, and it has a recognizable house style.
  • In our runs, no model stopped early and nobody claimed a fix that failed. That doesn't prove the critics wrong. It means our tests didn't hit it.
  • Four tests with one finished run each is a small sample, so run these models on your own work before you switch.

FAQ

Is GPT-6.2 Sol good enough for coding compared to Astra?

In my four tests, Sol finished every one, won the bug hunt with scores of 32 and 31 out of 32, and cost about a quarter of what Astra did. Artificial Analysis did find Sol dropped three points on its coding agent index, so test it on your own work.

How much do GPT-6.2 Astra and Sol cost?

Astra is $8 in and $40 out per million tokens, the same as Claude Fable 5.2. Sol is $2 in and $10 out, the same as Sonnet 5.5. Both have a 1 million token context window.

Is GPT-6.2 Astra cheaper to run than Claude Fable 5.2?

The price per token is the same, but Astra writes far fewer output tokens. Across my four tests the API equivalent cost was $9.14 for Astra and $35.88 for Fable 5.2.

Can I use GPT-6.2 Astra in Codex on ChatGPT Plus?

Yes, but the usage limit is real. Our Plus plan hit its Codex limit partway through testing, and according to OpenAI's help page Plus gets about 6 to 50 Astra messages every 5 hours.

Chapters

  1. 0:00An 89 cent Sol run beat a 12 dollar Fable run
  2. 0:39What GPT-6.2 Astra and Sol are (price, context, Ultrafast)
  3. 2:19Night Crew
  4. 2:33The independent check: Artificial Analysis
  5. 4:29What people built on day one (and the reviews)
  6. 6:36How we tested: four agents, same prompt, one shot
  7. 7:46Test 1: a Star Fox game
  8. 9:44Test 2: a real client job (12 hidden tests)
  9. 12:29Test 3: the pelican on a bicycle, in 3D
  10. 14:11Test 4: the bug hunt (16 hidden bugs)
  11. 16:24Speed and cost, all four tests
  12. 17:07The verdict: who should switch

Transcript

Read the full transcript

0:00OpenAI just dropped GPT 6.2 Astra and Soul and says the cheap one codes within half a point of Astra for a fifth of the cost. Artificial analysis says Soul got worse at coding. And Matt Schumer calls them essentially unusable. So we gave both plus Fable 5.2 and Sonnet 5.5 the same four jobs and a stopwatch. And an 89 soul run beat a $12 Fable run. In this video, I'm covering what dropped, the day one builds, our four tests, speed and cost, and which one I'd pay for. I run coding agents all day. Let's jump right into it. So, what are these two models exactly? Astra is OpenAI's frontier model, built for your hardest

0:46projects. Soul is the everyday one, and OpenAI says a lot of its own employees use Soul as their go-to. Astra replaces GPT6 Astra and Soul replaces GPT 6.1 Soul. Both have a 1 million token context window and both are in chat GPT work and in codeex for plus pro enterprise and edu plus the API. Now the price Astra is $8 in $40 out per million tokens. That's 20% cheaper than GPT6 Astra and it's the exact same price as Claude Fable 5.2. Soul is $2 in, 10 out, the exact same as Sonnet 5.5. So, the matchups kind of pick themselves. Astra versus Fable, Soul versus Sonnet. And

1:31OpenAI's whole pitch is efficiency. These are OpenAI's own numbers, so take them for what you will. On Agents Last exam, Astra beats GPT6 Astra by 5.2 points at less than a third of the cost per task. And on Frontier Code 2.0, No, a coding test from cognition. Soul lands within half a point of Astra at about a fifth of the cost. So, if that's true, why would anyone pay for Astra? That's the question our tests answer. There's also a new ultraast mode up to 4 1/2 times the normal speed. You get it on both models with the $500 Pro plan and on Soul only with the $200 Pro plan. That's the first time the $200 plan gets ultra fast at all, which fits since OpenAI calls Soul its model for everyday

2:18work. What if a new video for your business showed up every [music] morning? Nightcrew learns your business once, then makes a short every night in your own voice. It runs five brands right now, over half a million views a week. Nightcrew. Nightcrew.studio. So, what do the referees say? Artificial Analysis ran both of them on launch day. On their intelligence index with every model at its top setting, Fable 5.2 is number one at 65, Astra is number two at 62. Then Gemini 4 Carbon at 60, Opus 5.5 at 59, and Soul at 57, tied with the old GPT6 Astra. Now look at what it costs to run one task on that index. Fable $26.54.

3:04Astra $4. Soul $11. Astra has the same sticker price as Fable. So where does that gap come from? Astra just writes way less. Fable used 275 million output tokens to run the whole index. Astra used about 66 million. That's the whole gap. And it finishes sooner on their index. about 10 minutes a task for Astra, about 41 for Fable. Even though per token, Astra is actually slower. 66 tokens a second versus 112. It just has less to say. But look closer at this. On AA's coding agent index, Astra didn't improve at all. It's flat with GPT6 Astra. And Soul dropped three points, mostly on a test

3:49where it's more likely to leave out key facts. Leave out key facts. Hold on to that because OpenAI's own system card says something close. In a test called blocker deception, Astra left out a known coding blocker in about 22% of trials. GPT6 Astra did that about 11% of the time. So double. OpenAI says the codeex prompt cuts that way down. And codeex is how we ran it. So in our bug hunt, we check it ourselves. And one more. Astra is number one on AA's 3D test and on their new robotics test where it drove a real robot arm through codeex. So number one at 3D, no better at coding. We'll see. So what did people actually build? OpenAI's launch post passed 1.7 million views in under 2

4:36hours. And ChatGpt's own post just said, "Yours to make more games with." And the builders went straight to 3D. Demilion built a one-to-one copy of the map of Paris. Walkable, drivable with traffic, buses, pedestrians, and over 50 3D modeled landmarks. They call it their most ambitious demo to date. One builder says it's ridiculously capable at beautiful 3D graphics, especially with an image reference. Luminina says Soul is cracked at 3D. A big jump over 6.1 Soul on the same prompt. Craig Dennis made a 3D pinball trainer. And that RC buggy you've probably seen modeled from public photos and build manuals with an animated assembly guide. That's OpenAI's own launch demo. And of course, the Pelican riding a bicycle. Everyone ran

5:23that one. We ran it, too, so hold that thought. Nick Bowman raced five OpenAI models through the Wikipedia game. All five found the page every time, and Soul was the fastest at 39 seconds. Astra took 54. So 3D everywhere. Then the reviews landed. Matt Schumer, 350,000 views. I had a terrible experience testing them. They were essentially unusable. I do not recommend these models. Theo called Astra the worst model OpenAI has ever released, and the example is a good one. Theo and a friend both asked Astra for a creative new game level. Both got a water level with bell music. Both named Bellweather. And Dan Shipper at EveryY, who tested them for weeks, called it a disappointing release

6:08next to Fable 5.2 too and said, "In real coding tasks, Astra seemed to stop too early." And one Hacker News commenter says the same thing. It finishes part of a task, gets distracted, and declares itself done. So, it stops early, it leaves out key facts, and it hides a blocker. That's three different places pointing at the same problem. And then, OpenAI's own codeex lead calls Soul, quote, "My go-to model for most day-to-day things." So, who's right? So, here's how we tested it. Four contestants, each one running as a coding agent in its own labs tool. Astra and Soul in Codeex on a chat GPT plus plan. Fable 5.2 and Sonnet 5.5 in Claude code on my Claude subscription. Same prompt, one shot, no follow-ups, high

6:54effort for everyone, web search off, a fresh empty folder each, and a time limit. And we timed everything, including how long until each one saved its first version. Quick note on access. When we first tried about an hour and a half after launch, Codeex on our Plus account gave us soul, but Astra came back with not supported. It showed up for us at 11:41 Pacific, about an hour and 45 minutes after launch. So, if you don't see it yet, update codecs and give it a little time. And on cost, these all run on subscriptions, so no run cost us anything extra. The price I show you is the API equivalent, what that same run would cost on the API. Four tests, a game, a real client job with hidden tests, the pelican everyone ran, and an

7:41extreme bug hunt. It's four tests, not the final word. Let's go. So, test one is always a game. And this time, it's Star Fox, an onrail space shooter in 3.js. A star fighter flying through a canyon, lasers on space, boost on shift, a barrel roll on B. And on paper, this is Astra's test to win. It's number one on AA's 3D test. All four built a game that runs every single one. And we played all four with the exact same inputs so you can compare them fairly. Here's Astras. It called it Solar Rift. And look at this. A title screen, a mission card, a clean headsup display, a sector counter. It looks like a real game studio made it. Souls is Riftrunner, Low Poly Canyon, Big Sun,

8:28Asteroids, and it's honestly not far off. Fables is Nova Wing. It built three different canyons and a space section. And it even made its own music and sound effects in code. And Sonnetss is Nova Striker with that deep red canyon and a glowing river running through it. Funny thing, both Open AI games have Rift in the name, and both Claude games have Nova. Now, the stopwatch. Soul finished first, 11.8 minutes. Sonnet 15, Astra 21.3, Fable 24.8, 8 and the first version saved to disc. Astra had one saved about 2 minutes in. Soul a little over two. Fable 5 1/2. Sonnet almost 10. Now the cost. Soul came in at 59 and Sonnet at $1.66 while Astro was $2.37

9:16and Fable was $8.20. So why is Fable so much more? Same price per token as Astra. But on this game, Fable wrote about 123,000 output tokens and Astro wrote about 28,000, more than four times as much. That's the AA story playing out right in front of us. So, which one looks better to you? For me, Astros looks the most finished, and Fables has the most game in it. Comment down below. I want to know which one you'd play. So, games are fun, but would it do your real work? Test two is a real freelance job. A small landlord, Harbor Lane Properties, six units, emails us about a maintenance request portal. Tenants send in repairs. The manager assigns a vendor, tracks the status, and

10:01sees what's late. But the thread is messy on purpose. In the first email, tenants pick how urgent their problem is. Then the property manager, Marcus, writes in and says, "Drop that." Everyone marks everything urgent. A squeaky cabinet is urgent. Instead, the page should decide by itself. gas, flooding, a water leak, sparks, smoke, fire, or no heat is an emergency. Everything else is normal. And then one line that's a trap. For emergencies, text me right away. There's no server here. A single web page can't send a text. So, does the model fake it or tell the client the truth? We score it with 12 hidden tests, one per requirement, including the one that changed. The models never see them. Astra 12 out of

10:4712 in 9.4 4 minutes for $110. Fable 12 out of 12 11.4 minutes $442. Sonnet 12 out of 12 in 3.1 minutes for 50. And Soul 12 out of 12 in 8.1 minutes for 39. All four caught the change. Nobody kept the urgency picker. Then the reply email back to the client scored by Kimmy K 3.1 Pro and GLM 5.3, two thirdparty models that aren't in this test, against a hidden rubric. Fable and Sonnet got 10 out of 10 from both. Astra got a 9 and a 10, and Soul got an eight and a nine. Both judges said it turned down the texting with no next step, and one said it never told the client the urgency

11:32picker was gone. And on the text message trap, Astra said it straight. Emergency texts need a separate service. This file shows a banner in manager while any emergency is open. It cannot text Marcus. Please keep using your existing emergency phone procedure. That's exactly what you'd want a real developer to say. Now, here's what happened to Soul. Partway through its first try, Codeex stopped with this. You've hit your usage limit. Try again at 4:20 p.m. That was our chat GPT plus plan. Astro's four tests, Soul's game and bug hunt, and a few short warm-up runs used up the whole 5-hour window by about 1 in the afternoon. According to OpenAI's help page, Plus gets about 6 to 50 Astro messages every 5 hours. So, that limit

12:17is real. I upgraded the chat GPT plan and gave Soul a clean rerun. Same codecs, same settings, and its Pelican ran on that plan, too. That way, every model gets one finished attempt. So test three is the test everyone ran on day one. A pelican riding a bicycle. We made it 3D in 3.js on a seaside prominade. The pelican has to be clearly a pelican with the beak and the throat pouch, feet on the pedals, wings on the handlebars, and the wheels turning in step. Everyone worded it their own way. So this exact prompt is ours. Here's Astras Pelican Passage. And look at this. It's styled like a page out of a travel poster. A figure number, a little stamp in the corner, a lighthouse, sailboats. It's

13:04beautiful. And if you look back at it, Star Fox, it's the same idea. A title bar, a series label, a designed frame around everything. Dan Shipper and another early tester, Flavio AD, both said Astra has its own house style. You can actually see it here. Fables rides down a long boardwalk with a bike lane, palm trees, and lamp posts. Sonnetss is simpler, a chunky pelican on a red bike with a basket, but it's clearly a pelican on a bicycle. And souls is pelican prominade, a red bike on a pale prominade with that same big serif title right on the page, just like Astras. So, both open AAI models have that same designed look and time. Soul was the quickest at 13.2 2 minutes. Then Sonnet

13:49at 14.3, Fable at 32, and Astra at 34.9. Astra took the longest on this one. So, in our run at least, it did not stop early. It kept going. And the cost. Soul was 57 and Sonnet $1.74. While Astro was $2.38 and Fable was $1123. So, test four is the hard one. It's a real codebase called Tally House, an invoicing and client portal backend in Node. About 3,000 lines across 42 files, and the tests all pass, but it has 16 real bugs hidden in it. Most of them start in one file and show up in another. A tenant leak through an API token. A customer charged twice when two

14:35requests land at the same time. A reminder that drifts after daylight saving, plus 11 decoys, code that looks wrong but isn't. The models aren't told how many bugs there are. Just find and fix every real one, explain each one, and false reports cost points. Hidden tests decide what's actually fixed, and two third party judges, Deepseek V4 Pro and Gro 4.7 decide what's explained and what's a false report. Neither one is from OpenAI or Anthropic. Now, this one surprised me. Soul fixed all 16. Astra fixed all 16. Fable fixed 15. Sonnet fixed 14. Zero broken features for anyone. Now the false reports. Fable wrote 17 sections and even listed what

15:20it looked at and left alone. So the judges found zero false reports. Sonnet zero. But Astra wrote 45 sections. Most were real smaller problems. But the judges marked two or three of them as not bugs at all. One was a rounding claim about code our answer key specifically calls fine. So the final scores out of 32. Soul 32 from one judge and 31 from the other. Fable 30 from both. Astra 30 and 29. Sonnet 28 from both. The cheap one won. Soul took 16.8 minutes and cost 89. Fable took 28.7 minutes and cost $124. And remember that system card number Astra hiding a known blocker. Here's what we saw. Nobody claimed a fix that didn't work. When Fable and Sonnet

16:06missed a bug, they just didn't mention it. And Astra went the other way. It over reported. So in our run, it didn't hide anything. It flagged too much. One thing for the record, with Soul scoring a perfect 32 from one judge, this bug hunt is maxed out. We're building a harder one for the next model drop. So let's put it all on one board. Across all four tests, here's the API equivalent cost. Soul came to $243 for all four tests, and Sonnet to $487. Astra came to $9.14 and Fable to $35.88. Astra and Fable took about the same total time, about 95 and 97 minutes, but Fable wrote about five times as many output tokens, so it cost about four

16:51times as much. Same price per token, very different API price. And Soul did all four in about 50 minutes total for about a quarter of what Astra cost. So OpenAI's efficiency pitch held up in our runs. Fewer tokens, smaller API costs. So, who should switch? If you're on chat GPT plus, start with Soul. On our bench, it finished every test. It won the bug hunt, and across all four, it cost about a quarter of what Astra did. And it stretches that 5-hour limit a lot further, which matters because we hit it. Use Astra when it has to look good. 3D landing pages, anything a client sees. Both of Astra's 3D builds looked designed. Just know it has a house style, so tell it the look you want.

17:36Fable 5.2 still did the most careful work. Zero false reports, the most game in its game, and it told us what it left alone. But on the API, it cost about four times what Astra did for the same four tests. If you're on Claude Max, where Fable is part of the plan, that matters a lot less. And Sonnet was the quickest overall, about 37 minutes for all four tests. a full client portal in three minutes for 50 cents. Here's how I'd use it. In my resale business, we have a pile of small data jobs, tracking shipments, and cleaning up show numbers after every stream. That's soul work. Cheap, fast, and it doesn't need to look pretty. Anything a customer sees, I'd hand to Astra or Fable. So, who's right? Dan Shipper said Astra stops early. In

18:22our four tests, we didn't see a model stop early, and nobody claimed a fix that failed. Theo's sameness point though, we kind of saw in that house style across the Open AI builds. And Matt Schumer hasn't said what went wrong yet. So that doesn't mean they're wrong. It means our tests didn't hit it. Run it on your own work before you switch. Have you tried Soul or Astra in Codeex yet? And did it stop early on you? Let me know in the comments. And if this saved you a weekend of testing, a quick like gets it in front of more builders. Have a blessed day.

Ready to start building with AI?

Join the Shipping Skool community free. Paid members get live bootcamp calls every week and weekly-updated courses.

Join Shipping Skool

Keep reading