Gemini 4 Carbon vs Claude Opus 5.5: Benchmarks, Pricing, and Access
Prefer to watch? Watch on YouTube · Subscribe for new builds · Read the transcript
Google's new Gemini 4 Carbon just took the top spot on the Artificial Analysis Intelligence Index, and almost nobody can use it yet. In this video I compare Carbon with Claude Opus 5.5 number by number, explain why Google is giving it to cyber defenders first, and tell you who should care about it now and who should wait. It's for builders trying to work out whether this changes anything in their day-to-day stack.
What I cover in this video
- 0:00 Google's #1 model, and why you can't use it
- 0:46 What Gemini 4 Carbon is and how much it costs
- 2:18 The three-step rollout and the Fairwind Program
- 3:03 The scoreboard: Google's table vs the independent referees
- 6:06 The cyber story: find, exploit, patch
- 9:10 Honesty numbers and the 100-second wait for the first token
- 10:41 What people built with it in Arena's preview
- 12:11 Who should care now and who should wait
What Gemini 4 Carbon is and how you get it
Carbon is Google DeepMind's new frontier model. It came out two weeks after Gemini 4 Argon went public. Google calls it their strongest model and says it's state-of-the-art at cybersecurity and long-horizon tasks, meaning it can stick with one task for a long time. Google says they've seen it run for days internally. That's their claim, and we can't check it. Artificial Analysis lists it with a 1 million token context window.
The introductory price is $2 in / $10 out per million tokens, the same as Argon. After that it goes to $4 / $20, and Google doesn't say when the intro period ends. Opus 5.5 is $4 / $20, so Carbon costs half as much during the intro period and the same afterward.
Right now, though, you can't get it. The rollout has three steps:
- A subset of cyber defenders, starting this week.
- More trusted partners through the Fairwind Program, Google's security program with over 650 partners such as governments and critical infrastructure.
- Developers, businesses, and consumers "as soon as possible," with no date.
Google also says it's taking part in the US government's voluntary pre-release testing. The only precedent we have is Argon: it was announced September 30th, went to cyber defenders first, and opened to everyone on October 13th. That doesn't mean Carbon will follow the same schedule.
The scoreboard: Carbon vs Opus 5.5
Google's own table says Carbon is best on 15 of 17 benchmarks. One of the two it loses goes to GPT-6 Astra. The other is OSWorld (computer use), where Opus 5.5 wins 87.2% to 81.5%. Opus also edges Carbon on FrontierSWE 2, 62.3% to 61.2%. Keep in mind that Google's fine print says the other labs' numbers are mostly self-reported.
Next, the referees, where someone else runs the tests:
- Artificial Analysis Intelligence Index: Carbon 59, Opus 5.5 (max) 58, Argon 50. Carbon costs $7.76 per index task vs $10.04 for Opus, about 23% less, and uses 29% fewer output tokens.
- Coding Agent Index: Opus 5.5 in Claude Code 62, Carbon in Google's Antigravity CLI 59, Argon 53. Carbon finishes a task in about 30 minutes for about $7.74. Opus takes about an hour and costs $11.42.
- Vals Index: Sonnet 5.5 70.85%, Carbon 70.36%, Opus 5.5 70.18%. They're all inside each other's margin of error, so it's basically a three-way tie.
- Arena: #1 in Text for now, #4 in WebDev (Opus 5.5 max is #1), and #2 in Agent Arena behind Opus. At launch it was 72% cheaper per task than Opus on max.
On knowledge work like spreadsheets and documents, Claude models are about 200 points ahead on Artificial Analysis's briefcase test. On long context, Google's table shows Carbon at 96.2 vs Opus at 66.8 on Graphwalks, but that number comes from Google.
So Carbon wins the overall index by one point and is cheaper and faster per task. For the coding work builders do every day, Opus is still on top. I use Opus 5.5 daily in Claude Code, and nothing here tells me to switch today.
Why it's locked up: the cyber story
A vulnerability is a mistake in code that lets someone do something they shouldn't. A zero-day is one nobody knows about yet, so no fix exists. In security, a model can do three things:
- Find: Carbon scored 88.2% on SEC-bench Pro. There's no Claude score on that chart.
- Exploit: On ExploitGym it scored 54.8% with 10 hours per run (Argon got 42 with the same time). With a 6-hour limit, Carbon scored 53.5% and Opus 5.5 scored 34.5%.
- Patch: CWE-bench, Carbon 71% vs Opus 67%.
Google says these were run without guardrails, so this is the raw model. Google's blog also says Wiz sees Carbon leading on web penetration testing. Someone who ran offensive AI at Wiz said it helped them find and patch critical zero-days, but their bio says they now work at Google, so they aren't a neutral source. The Chrome team says it was more than 10x better than another lab's frontier model at writing patches, and they don't name the lab.
My read: the skill that finds and patches a hole can also break in through it. Google never says Carbon is too dangerous to release. Its Fairwind page does say that in the wrong hands, the same capabilities can be an equally powerful threat. For small businesses like our resale business, which runs on apps I built with Claude Code that nobody security-audits, a model that finds holes first would be a big deal.
Honesty, speed, and what people built
Google says Carbon makes about half as many unsupported claims as Argon, with overreach down 26% and under-service down 29%. It also reports 0.0% on Gray Swan's prompt injection test and 0.0% successful cheating on Terminal Bench. The Decoder pointed out that the prompt injection test looks saturated, and all of these numbers come from Google's own testing. Artificial Analysis's hallucination test found Carbon went up, from 4% for Argon to 11%. That's still under a quarter of Opus 5.5 max, which sits around 50%. These are different tests, so both results can be true.
Speed: Artificial Analysis measured 100 seconds to the first token for Carbon. Opus 5.5 on max takes 664 seconds.
The Carbon outputs you've seen on X came from Arena's preview, which ran pre-release checkpoints. We couldn't run Carbon ourselves. People built a 3D White House, London from Roman times to the Blitz, black holes, and an airliner that breaks apart piece by piece. It didn't win everything, though: Opus took the poster test. These models are getting good at 3D fast, but a checkpoint on Arena isn't the same as a real agent run.
Key takeaways
- Carbon is #1 on the Artificial Analysis Intelligence Index by one point, not the blowout Google's 15-of-17 table suggests.
- Opus 5.5 still leads on coding agents, WebDev, Agent Arena, and knowledge work. Carbon is faster and cheaper per task.
- Security teams, governments, and critical infrastructure should look at the Fairwind Program, which is the way in right now.
- Builders should wait. Argon is available today on OpenRouter at $2 / $10 if you want to see where Google is headed.
- When the API opens, we'll bench Carbon in Google's agent against Opus 5.5 in Claude Code on the same tasks with hidden tests. More on AI agents.
FAQ
Can I use Gemini 4 Carbon right now?
Not yet, unless you're one of the cyber defenders Google picked or a Fairwind Program partner. Google says developers, businesses, and consumers will get it as soon as possible but hasn't given a date.
Is Gemini 4 Carbon better than Claude Opus 5.5?
It leads the Artificial Analysis Intelligence Index 59 to 58. On the Coding Agent Index, Opus 5.5 in Claude Code still wins 62 to 59, though Carbon finishes tasks in about half the time and for less money.
How much does Gemini 4 Carbon cost?
The introductory price is $2 per million input tokens and $10 per million output tokens. After that it rises to $4 / $20, the same as Opus 5.5, and Google hasn't said when the intro period ends.
Why is Google releasing Gemini 4 Carbon to cyber defenders first?
Carbon scores very high at finding, exploiting, and patching vulnerabilities, and the same skills could be used to attack systems. Google calls it a deliberate rollout while it keeps working on guardrails.
Chapters
- 0:00Google's #1 model... and you can't use it
- 1:00What Gemini 4 Carbon is (and how the rollout works)
- 2:54Join Shipping Skool
- 3:14The scoreboard: Carbon vs Opus 5.5
- 6:17The cyber story: find, exploit, patch
- 8:58The honesty numbers (and the 100-second wait)
- 10:28What people are building with it
- 12:00Who should care now, and who should wait
Transcript
Read the full transcript
0:00Google's new model just went number one on one of the biggest independent AI leaderboards and you can't use it. Gemini 4 Carbon just took the top spot on the artificial analysis intelligence index with a 59. That's one point over Claude Opus 5.5 on max effort. One point. But here's the thing. Google's own blog says it's going to a subset of cyber defenders first. No date for the rest of us. Sundar Pachai basically said, "Hang tight." And on the independent coding agent scoreboard, the one that tests real coding work, Opus 5.5 in Claude Code is still on top, 62 to 59. So, in this video, I'm going to be breaking down what Carbon actually is, every number head-to-head against
0:46Opus 5.5, why it's starting with cyber defenders and what it can actually do in security, the honesty numbers, what people are already building with it, and who should actually care right now versus who should wait. Let's jump right into it. So, what is Gemini 4 Carbon? It's Google DeepMind's new frontier model, and it dropped just 2 weeks after Gemini 4 Argon went public. Logan Kilpatrick called it a big jump over Argon, which we released just two weeks ago. And if you watch my Argon video, you know I was not impressed. Like I said back then, vendor reported Google ran Argon itself. Big mistake. Google calls Carbon its strongest model. State-of-the-art at cyber security and long horizon tasks, meaning it can keep
1:31working on one task for a really long time. Google says internally they've seen carbon run for days pushing through dead ends. That's their claim. We can't check it. Artificial analysis lists it with a 1 million token context window. And the price is the same introductory price as Argon. $2 in, $10 out per million tokens. After the intro period, that goes to $4 in, $20 out. And Google doesn't say when the intro period ends. For comparison, Opus 5.5 is $4 in, $20 out. So, at the intro price, carbon is half the price of Opus. After that, it's the same price. So, how do you get it? You don't. Not yet. Here's how the roll out works. Step one, starting this week, a subset of cyber defenders. Step two,
2:18more trusted partners through something called the Fairwind program. That's Google's security program with over 650 partners like governments and critical infrastructure. Step three, developers, businesses, and consumers as soon as possible. no date. And on top of that, Google says it's taking part in the US government's voluntary pre-release testing process. And here's the funny part. Argon did the exact same thing. It was announced September 30th, Cyber Defenders 1st, and it opened up to everybody on October 13th. That doesn't mean carbon follows the same schedule, but it's the only precedent we've got. Real quick, every video I make is built in code with Claude Code. Inside Shipping School Premium, I show you how step by step to turn your product into
3:03videos that get views, plus new courses every day, 500 plus hours of replays, live calls every week, and a Q&A with me every Saturday. Join free link below. So, let's do the scoreboard. Carbon versus Opus 5.5 head-to-head. First, Google's own table. 17 benchmarks and Carbon is the best on 15 of them, which sounds amazing. But look at the two it loses. One goes to GPT6 Astra and the other one OSWorld, which is Computer Use, goes to Opus 5.5. 87.2 versus 81.5. And if you go head-to-head, Opus also edges carbon on Frontier SU 62.3 to 61.2. And here's something to be wary
3:48about. In Google's own fine print, the other lab's numbers are mostly self-reported. So take that for what you will. So, let's go to the referees. These are the independent scoreboards where somebody else runs the test. Artificial analysis intelligence index. Carbon 59. Opus 5.5 on max 58. Argon was 50. So, that's a 9point jump since Argon, which is insane. And it's cheaper than Opus on Max. $7.76 per index task versus $104. About 23% less. and it uses 29% fewer output tokens to get there. Now, the one I actually care about, the artificial analysis coding agent index. This tests the model inside a real coding agent.
4:35Opus 5.5 in Claude Code 62. Carbon in Google's anti-gravity CLI 59. Argon 53. So, Opus still wins. But to be fair, Carbon finishes a task in about 30 minutes. Opus takes about an hour and carbon costs about $7.74 a task versus 11.42. So Opus is better. Carbon is faster and cheaper. Valai Sonnet 5.5 actually tops it by half a point at 70.85. Carbon 70.36, Opus 70.18. All three are inside each other's margin of error. So that one's basically a three-way tie. And then Arena, where real people vote. Carbon is number one for now in text arena. But in webdev,
5:21where you build websites, it's number four. Opus 5.5 on max is number one. And in agent arena, carbon is number two, right behind Opus. But Arena says at launch, it was 72% cheaper per task than Opus on Max. 72%. And one more. On knowledge work like spreadsheets and documents, the claude models are about 200 points ahead on artificial analysis's briefcase test. And the long context number is wild. Graph walks, which tests reading 256,000 up to a million tokens. Carbon 96.2 opus 66.8. That's from Google's table though, so same grain of salt. So you can see the pattern. Carbon wins the overall
6:06index by a point. And against Opus on Macs, it's cheaper and faster per task. But the coding work that builders actually do every day, Opus is still on top. So now the big one, why is Google keeping this thing locked up? It comes down to cyber. So first, what's a vulnerability? It's basically a mistake in code that lets somebody do something they shouldn't, like read your database or take over your account. And a zero day is one nobody knows about yet, so there's no fix for it. And there's basically three things a model does in security. Find the bug, exploit the bug, and patch the bug. Step one, find. SEC Bench Pro tests whether a model can find real vulnerabilities in real code. Carbon scored 88.2%.
6:53Number one on Google's chart, although there's no Claude score on that one. Step two, exploit. Exploit Gym tests whether it can actually break in. With 10 hours per run, Carbon scored 54.8%. 8%. Argon with the same 10 hours, 42. And at the same 6-h hour limit, it's Carbon 53.5 versus Opus 5.5 at 34.5. So, Carbon is way ahead on the scary one. Step three, patch. C.WE tests whether it can write the fix. Carbon 71%. Opus 67. And here's something to be wary about. Google says these were run without guard rails. Mitigations off. So this is the raw model. Google also says Whiz, which
7:39its blog calls its own cloud security platform, sees Carbon leading on web penetration testing. And Naggi, who runs offensive AI at Whiz, posted that it already helped him identify and patch several critical zeroday vulnerabilities. Now, to be fair, his bio says he's now at Google working on Gemini Cyber, so he's not exactly a neutral voice. That's Google's own people, not an outside test. And think about what that means for a small business. In our resale business, we run on apps we built ourselves with claude code. Nobody's doing a security audit on a Chrome extension I built in 20 minutes. A model that finds the holes before somebody else does. That's actually huge. Google's Chrome team also says it was more than 10 times better than another lab's big frontier model at
8:25writing highquality patches. They don't say which lab, so grain of salt. So here's my read. The same skill that finds a hole and patches it can also break in through it. That's why defenders get it first. Google's own words are a deliberate approach to roll out and they're still iterating on guard rails. They never say it's too dangerous to release, but their fair wind page says it straight. In the wrong hands, the same capabilities can become an equally powerful threat. So, it's pretty obvious why the security people are first in line. So, let's talk about honesty because Google spent a lot of the blog on this. Google says Carbon makes about half as many unsupported claims as Argon. Overreach is down 26%
9:10and under service is down 29%. On Grey Swan's prompt injection test, Argon got tricked.7% of the time. Carbon 0.0. And on Terminal Bench, successful cheating went to 0.0%. Now, the decoder pointed out that the prompt injection test looks saturated. And all of this is Google's own testing. And here's where it gets interesting. Artificial analysis ran its own hallucination test. And Carbon's hallucination rate actually went up 4% on Argon, 11% on Carbon. To be fair, that's still under a quarter of Opus 5.5 on Max, which sits around 50%. Now, those are different tests, so both can be true. Google measured unsupported claims in its own testing and artificial
9:55analysis measured how often it makes stuff up instead of saying I don't know. But that's exactly why you want somebody else checking the vendor's numbers and the speed. Artificial analysis clocked Carbon at 100 seconds to the first token. The typical model in its class is under 4 seconds. So, you're going to be waiting. But before anybody makes fun of it, Opus 5.5 on max takes 664 seconds. And even on X high, it's about 109. Remember, Opus is called Slopus. It has been slow since the dawn of time. So, what are people actually doing with it? And you're probably asking, if nobody can use it, how is everybody on X already posting carbon outputs? That's Arena. Google gave Arena preview access,
10:41and a carbon checkpoint was sitting on Arena for about a week before the announcement. So, keep in mind those are pre-release checkpoints, not necessarily the exact model Google ships. Quick disclaimer, we couldn't run Carbon ourselves. Every Carbon clip you're about to see came out of Arena. Arena's Peter Gostev put it through a 47minute run. Same prompt, three models, Carbon, Argon, and Opus 5.5, building the White House in 3D. And then he had carbon build London through the years from Roman times all the way to the Blitz, which is pretty awesome. And to be fair, it didn't win everything. On the poster test, Opus 5.5 took it. Then you've got Luminina Bench with a helicopter test against Opus. Carbon was done in arena
11:26in a little over 30 minutes. Opus took about 55 in Claude Code. Mr. Salio had Carbon build an airliner that breaks apart piece by piece right next to Opus' jet. And people are posting black holes, a Bentley driving through the sunset, all of it. One poster even said that once it officially drops, Opus 5.5 is in serious trouble. I mean, come on now. It's an arena checkpoint. Let's see it in a real agent first. I mean, is this going to change the world? No. But these models are getting really good at 3D really fast. So, all in all, here's my take. Google's table says 15 out of 17. The referees say one point overall. And Opus still wins coding agents, webdev,
12:11agent arena, and knowledge work. So, it's really good. It's a huge jump from Argon, but it's not the blowout the vendor table makes it look like. So, who should care right now? Security teams. If you're a defender, a government, or critical infrastructure, look at the Fairwind program because that's the way in right now. Who should wait? Builders like me. You can't use it anyway. In the meantime, Argon is out right now on Open Router at that same $2 in, $10 out. So, if you want to see where Google is headed, that's the one you can actually call today. I use Opus 5.5 every single day as my daily driver in Claude Code on my $200 a month plan, and nothing on this scoreboard tells me to switch today. But here's what I'm watching. If Carbon opens up at $2 in, $10 out, and
12:58it finishes coding tasks in half the time, that's a real option. So, the day the API opens, we're going to run our own bench. Carbon in Google's own agent versus Opus 5.5 in Claude Code. Same tasks, hidden tests, and we'll see if the numbers hold up. Quick gut check for you. After Argon, do you still trust Google's own benchmark table? Yes or no? Give me your answer in the replies, and while you're down there, throw a like on the video. Have a blessed day.
Ready to start building with AI?
Join the Shipping Skool community free. Paid members get live bootcamp calls every week and weekly-updated courses.
Join Shipping SkoolKeep reading
Gemini 4 Carbon vs Claude Opus 5.5: Benchmarks and Speed Test
Gemini 4 Carbon tied Claude Opus 5.5 on both of my hidden-test builds (12/12 and 28/32 each), but in my runs through OpenRouter it took up to 7.6 times longer and twice saved no file. Carbon costs $2 in / $10 out until November 30, then the same as Opus.
DeepSeek V4.1 Flash Explained: Pricing, Benchmarks, and Claude Code
DeepSeek V4.1 Flash costs $0.15 in and $0.60 out per million tokens off-peak. Artificial Analysis scores it 32, ahead of DeepSeek's own V4-Pro at 23 but well behind Claude Opus 5.5 at 58. I'd use it with Claude Code for cheap bulk chores and second-opinion bug hunting, not for hard reasoning or private data.
Kimi K3.1 Pro vs Opus 5.5 and Sonnet 5.5: Benchmarks, Cost, Tests
Kimi K3.1 Pro costs the same per token as Sonnet 5.5 but writes far less, so it works out about 2.5x cheaper per task. It finished all three of my one-shot tests for 32 cents total. Opus 5.5 still scores higher and is still my daily driver.
