Fable 5 vs GPT-5.6 - It Wasn't Close

0:00There's been a ton of hype around GPT 5.6 release lately and yesterday I was testing it meticulously all day long running it through particular benchmarks that I actually use in my business in production and writing code and fixing bugs. So in this video we're going to be going through it all. I have a few benchmarks here. You see that Soul Ultra test really well at a third of the price of Fable 5 and as we know Fable 5 in 2 days is going to be off of your subscription plan. So Anthropic is really in a hard position here. Are they going to continue to allow users to use it in the subscription or are you going to be paying for usage base? So you can see here even Codex Terra 5.6 on max

0:45outscores Fable 5 with fallback and you can see here also cursor on the cursor bench. You can see here that a third of the price of Soul max comparable to Fable max and the tokens per task and the cost per task is literally three times as much and the output according to others people's tests has been pretty decent but does it stack up to Fable 5? That's what I want to know. And you can see here on terminal bench as well terminal bench 2.1. This has been out for about a week now but you can see here Soul Ultra and even GPT Soul is outcompeting Mythos and Fable 5 by a good 5-10%. So that's why it's really important to actually use these models

1:32for your use case to then create an actual opinion because the benchmarks a lot of the times are benchmarks by a lot of these big companies to create a ton of hype to allow you to use the models and then when you use the model you are you know felt with like kind of like rug pulled, right? So we need to actually use these models in production to see if they're even worth using, right? So we're going to be testing out a few things. We're going to be creating a landing page. We're going to be creating a few games. And today's models that we're testing out is Fable 5 Max versus GPT 5.6 Soul Ultra. And we're just going to give the flat results. We're going to copy each prompt, put it into two different the Codex CLI and Cloud Code

2:17CLI. And we're going to just let them rip. And we're going to see these tests come out and then we'll be able to formulate opinion which one do I want to use? And is it worth paying for Fable? If it's worth paying for Fable and it saves me more time, then I think it's worth the upgrade. But if Soul Ultra does just as good of a job, we'll let you decide. Let's get right into it. Here's the specific run sheet of some of the tests that we're going to be running with Soul Ultra and Fable 5 Max just to see if Soul 5.6 Ultra can keep up with Fable 5 Max. And I'm referring to the effort level. So, Fable 5 Max is going to be the highest reasoning effort level and then Soul Ultra is going to be the highest effort level for OpenAI. So, this is going to be just fresh session

3:02per model per test. We're going to paste the prompts in verbatim and they're going to one-shot these tests. So, we're going to work through a few of these. We'll go through the lava lamp test. We'll have it create a video game. We'll have it go through a debug gauntlet. We'll also put it through its paces on how well each of these models can produce a remotion video as well as a blender build. So, this is going to be interesting. Let's get right into it and kick off these test results. All right. So, what we're doing here is we are actually running the gauntlet right now here inside of Beekin. And basically this is a voice-controlled ADE. And we're going to be able to see these test results happen in real time. It's going to spin up basically a Soul agent running on Ultra and it's going to spin up a Fable 5 Max agent. So, these are

3:50going to be able to run through these test results, and we will receive it happen in actual real time inside of the back-end bench workspace. So, no panes exist yet in the workspace. It's going to open the two terminals in the arena and start working on it now. I'm having Fable 5 on extra high be the producer and basically manage all of these test results. So, it just opened two new panes here here, two new terminals, and it's going to basically call the Codex agent in the Cloud Code agent via the MCP. And see, it had labeled It had labeled the terminal panes Soul and Fable. That way we could keep it more organized. And I'm going to put it on auto mode. Hey Becca, can you tell Cedar the terminals are all set up so it could

4:37go ahead and pass those prompts along to Soul and Fable terminal panes. >> Right. Let me pass that along to Cedar now. Sent to Cedar. >> And you can see here it is now running the tests on both Soul and Fable. We're on Soul 5.6 ultra right here. You can see the reasoning level. And then you can see here we are on Fable 5 Max. And we're on auto mode. So, we are basically the prompt is the identical. It says, "Draw a self-portrait of yourself as an AI. A single SVG file. Output only the SVG code. No explanation. Be ambitious. Composition, color, lighting, detail. It should feel like you, not a generic robot." This is going to be a good one.

5:22And you can see here test four is rolling. This is Cedar, my Fable agent that is basically the producer and orchestrator of this test. So, it's on standby watching the arena when both models stop, and it will just tell me and I'll tell Cedar it will collect the results, and then it will pull up the output here in this file, and we'll check it out. And then we'll start test number two. Real quick, Soul has finished minutes ago, and Fable 5 Max is still going 7 8 minutes in. Soul finished its output about 5 minutes ago. Let's see if it's better. One of the cool things about Backen is that I can switch between different workspaces on

6:08what I'm working on, what I want to do. I can go to tests, I can go to Backen marketing. There's different ways that you could work in different workspaces all connected via MCP, which is awesome. So, it allows you to basically do multiple things at once with hundreds of agents all communicating each other on a workbench via MCP, all based on the Google open knowledge format and ICM frameworks, which is basically the folder is the framework. They're working out of basically markdown files that are organized, and each agent can communicate using the YAML front matter. So, this is an amazing project I've been working on for the last month, and it's made building AI apps in general so much

6:54better, so much easier. I was tired of jumping around from terminal to terminal, figuring out what was I working on. This way, it just keeps it much more clean and easy to spin up a bench arena just like this, just like what we're looking at. Even though Fable 5 Max is taking 11 minutes to do a simple SVG. What is going on? It better be good. Okay, wow, 17 minutes. That's how long Fable 5 Max took compared to Soul, 3 minutes. So, okay. Hey Backen, um so, can you tell Cedar the results are done? And wow, Fable 5 Max took forever. Can you pull Can you tell Cedar to pull the results up inside of a web browser, and I can check them out?

7:40>> All right, I'll pass that along to Cedar so it can bring the results up for review. Said to Ceda. >> All right, so we're going to check this out and then we're going to move on to the next test results or the next test after this. Results are in and I know it's such a simple test, but you could see JBT 5.6 Soul Ultra here, the clock was 3 minutes and um wow, um the output tokens was about 10,000. And Fable 5 Max almost 18 minutes, output tokens 190,436. Wow, oh my goodness, this is uh pretty incredible. Um so let's take a look. Um here is the artifacts.

8:26Um I I I really don't know what to think. I I don't know. Um yeah, it says Fable 5 copied exactly as written to its arena folder. Um holy smokes. Okay, let's check it out. Just going to open the results folder and just see how it did with the output. I cannot see the SVG. Is it possible to pop open two web browser previews with the screenshots? Here is the screenshot of what I see when I try to open it inside of a folder. Okay, here we go. We got both outputted here. I don't know what to say. Claude Fable 5 Max took what? Six times as long and the Soul

9:11Ultra 10,000 output tokens versus 10 times the output tokens. That is crazy and I think that honestly I like the Soul Ultra variation much better. It has animation. Fable 5 Max has no animation. Little more complex, but holy smokes. Like I don't know. I don't I don't it's worth it on that on that, you know. Um a library dreaming it is a reader. Okay, interesting interesting choice Claude. Interesting choice Claude. Okay, on to the next test. We are going to Hey Becca, can you tell Cedar on to the next test? I think Soul Ultra won that round. Not only just with the output tokens,

9:57but the time it took. Fable 5 Max took way too long and there was no animation in the SVG. Okay, so can you tell Cedar that let's move on to the next test. >> All right, I'll pass that along to Cedar now. Fact Cedar. >> So per the rules, we are now starting a fresh session with a fresh new test and we're going to go ahead and move on to the lava lamp test. Which is going to be a good one. So we're going to go ahead and just get these on auto mode and get them cooking. Here's the prompt being passed along. Create a realistic lava lamp simulation of a self-contained HTML file. No external libraries or assets. Blobs of wax heat up at the bottom, rise, cool at the top, sink, and deform, merge, and split like real wax. Get the

10:44slow hypnotic pacing right. It should be mesmerizing to watch for a full minute. It must work simply by opening the file in a browser. Boom, those are kicked off and running. So far Soul Ultra is in the lead in my opinion with that first test. What do you guys think? Drop in the comments. Have you been using Soul Ultra or sticking with Fable 5? I think Fable 5 for the quality, I think it's still worth it if you have the time and the money to spend. But like I said, in 2 days Fable 5 is going to come off of your subscription and you're going to have to pay usage base credits. So so far, I think Soul Ultra is in the lead just from the SVG test. We're going to be going through this lava lamp test just to see how it does. Okay, so that took 56 minutes. Wow. And Fable 5 Max has been

11:32going through this API error, still going, 58 minutes. Um but nonetheless, ChatGPT 5.6 Soul Ultra is done. And we're going to check it out. Can you ask Cedar what the status is on the lava lamp update? And can you pull them up in a web browser within the workspace their output? >> Right. I'll check with Cedar and open a browser here so it's easy to follow. Sent to Cedar. The lava lamp update is still in progress. Test 02 is running in both panes and Cedar plans to collect and view both lamps once they finish. A browser is open in the workspace. >> I mean, it's okay. Look at the output here. We'll need to

12:17see what Fable did just to be able to compare. Okay, quick little summary. Soul Ultra finished at 56 minutes here and you could see here it produced the HTML and Fable 5 Max is still iterating, which is insane. It's doing max effort things, that's for sure. Both are dramatically past the tests 04's pace. This is a slow burn on both sides and we will wait for Fable's output. So far, Soul Ultra's winning, but not by much. Next, we're going to be doing a UI test. We're going to have it recreate my landing page and see if it can do a better job. So far, just like going through this process, it's taking a while on Fable Max effort and it's

13:02producing a lot of output tokens to do something simply that Soul Ultra can do. So, if Soul Ultra can perform this much quicker and give you 90% of the quality of the output, why wouldn't you just use Soul Ultra versus Fable 5 Max? If you were using Fable 5 Max for maybe long-running tasks or running a loop. This is something that maybe you would consider, but right now, I mean, it's just a simple lava lamp HTML file. Something that can be produced much quicker than 1 hour and 6 minutes. So, that is kind of my thoughts as we go through this testing process. What are your thoughts? Have you used Fable 5

13:47Max? I did for the first release when it came back on July 7th, and it performed well. It did take a long time on things that were just unnecessary. So, I think that if you're using even extra high or max while using Fable 5, I really feel like that is so unnecessary for simple tasks. Now, if you're trying to want to refactor your codebase or look for security vulnerabilities and you have time or you want to set it on a loop, then maybe that's something where you would consider running Fable 5, you know, at that higher effort level. But, I think medium to high effort level for Fable has been kicking butt. I have been using it in my other workspaces on medium to high and it's performed so much better than Opus 4.8, by the way. It just gets

14:34it. It gets the task. I clear the context window. We move on to the next feature. And it's been much quicker than 1 hour and 7 minutes. So, while we're still waiting, those are just my thoughts. The next test I'd like to do is have it recreate my landing page here for Hey Beacon. This is something that I built out with Opus 4.8. It was tedious, but at the same time, I think that just having it one-shot it to be able to at least see, you know, how well it can do and create a remotion video to put here as well. So, it can be like two tests in one and have these parallax effects. I think, you know, let's This is a good shot, right? This is a good shot to try and one-shot it. So, let's run that as the next test after this after we score

15:20up the lava lamp game. While it did take a long time to produce these results, I think that Soul Ultra did a better as far as the physics and Cloud Fable 5 Max took literally 10 times the output tokens as Soul Ultra here. You could see here they both took about an hour. Fable 5 Max took longer than an hour, but took up 676,000 output tokens versus Soul Ultra, which just took 67,000 tokens on the output. Pretty remarkable. I What do you think? What do you think? I feel like a lot of influencers have come out in the last few days since GPT 5.6 Ultra release and

16:05all saying it's amazing, but I'm really really trying to grasp, is it for real? Because when you are paid for a sponsorship and nothing against that, you have a general bias towards a product, right? So, while I think Fable 5 Max the output is actually a bit better, I mean, I'm going with Soul Ultra every single time because it used 10 times less the output credits. That's insane. I'm going to move on to the next test, which is going to be a UI test where it's going to recreate my landing page. Let's see how it does because GPT in general, 5.5, GPT 5, has always done really well with back end, but as far as working on design, it's been terrible. It has been absolutely awful. If you

16:51look at the design arena benchmark results, it's always been really poor as far as UI, front end, design with building any type of landing page, any type of wireframe. So, I'm really interested to see if these two models can just one-shot my landing page in one go. Let's get going and tell Cedar to get started on this next step. Can you tell Cedar that I am ready for the next gauntlet test? Can we pick the one where it is a UI design or a landing page design? I preferably like them to just see if they can recreate my heybacon.ai landing page with one shot with one prompt. Let's give this a go. >> All right, I'll pass that gauntlet test prompt over to Cedar now. Sent to Cedar.

17:36>> Okay, so here's the prompt. Recreate the landing page shown in the reference PNG reference 2 PNG. So it took screenshots in this folder. Four consecutive full-width sections on one page top to bottom captured. So it's going to try and recreate heybacon.ai, which is the landing page for our product. So this is going to be one shot completely controlled by Cedar agent kind of guiding them through the process. So we got Fable 5 here on set to max and then we have Soul Ultra going at it head-to-head. Let's see how long this one takes. Here's a quick little update. The exhibition stats on both models are 27 minutes in, which is kind of crazy, but they're still working.

18:22Soul appears to have spawned sub-agents and it's waiting on them. Fable is at 112,000 tokens and climbing and it's going to collect when they stop, which is Cedar our agent here basically controlling the whole benchmark. So pretty interesting. Like I said, most people are not going to use Fable Max or Soul Ultra. I just want to know if I'm running a loop and I want to run it overnight and I have unlimited token spend. I want to know what model is going to give me the best results regardless of tokens and cost. That's why we're running these benchmarks. One thing worth noting when you're using Soul on ultra effort level, when it spawns sub-agents, it will also have those sub-agents run on ultra.

19:08People have been coming out and complaining, "Wow, my $200 OpenAI plan is not as token efficient as it once was maybe a month ago. You can get a ton of subsidized tokens using GPT 5.5 or GPT 5.4 in Codex running long running tasks and you would hit maybe 2% weekly usage. Now, that doesn't go as far. So, something to think about and consider when you're spawning sub agents and same goes for Fable 5 and you shouldn't really be spawning sub agents to begin with when you're using max or extra high effort levels. Okay, so it finished. It finished, it's done. Amazing. We are going to now compare side by side because Soul finished literally in about

19:5425 minutes and Fable 5 it finished in about 45 minutes. But, we need to see the output and how much token were spent because the lava lamp test technically was about $35 if you were paying per token on Fable 5 and is about $6 when you're comparing it to Soul's output. So, you really need to count the costs. Is this worth it for the work that I'm doing? And like I said, 2 days, the time is ticking where Fable 5 is supposedly being taken off our subscription. So, we need to make a decision. Are we going to be switching to Soul 5.6 for most of our tasks? That has yet to remain to be seen. Okay, the results are in and I am

20:40pretty impressed on Soul Ultra, guys. Like, no joke. This is interesting and I'm glad I'm running these tests just to show you the raw and real. I don't consider myself an influencer. I wasn't paid by anyone. I just want to see how can I get the most token spend for the least amount of money, right? How can I get the best output for the least amount of money? And so far, Soul Ultra, yes, it does take an extremely amount of time. Like, 40 minutes, that's that's a while. But, you can see here at 199,000 tokens out, 40 minutes, and then also it had a 14-minute outage, by the way. So, what, let's say 26 minutes versus Fable 5 Max at 45 minutes, almost double the amount of time, and triple the amount of tokens. That means that this landing

21:26page recreation, very good by the way, is $35. It's $30 if you were to pay usage. It did a great job. It did not apply the parallax effect. Here is the original right here, heybacon.ai, and you can see I have a parallax effect that the demo video then plays in the first scroll, and then it parallaxes into the features. But, very interesting, very, very interesting, Soul Ultra has impressed me. What are your thoughts? I wanted to make this review just kind of showing you through the process of kind of what is it all about. I think, in my opinion, if Fable 5 ends up coming off the subscription, obviously everyone is just going to switch to Soul. You don't need to to use

22:11it on Ultra, but as far as a cost-effective way to build, to vibe code your next app, to fix bugs, to refactor your code base, that's going to be the obvious winner and my choice. Leave in the comments down below, please let me know your thoughts. What do you think about these new model drops? I hear Opus 5 is on the way. I hear Fable 5.1 is already around the corner. This is a crazy time to be alive, but we knew we do need to be conscious on how much we're spending when it comes to token costs, because AI and using AI is not going to get cheaper. We'll see in the next one. Have a blessed day.