Watch Me Benchmark Fable 5 vs GPT-5.6

0:00We need to do more testing when it comes to Fable 5 uh versus Soul. And in this video, we're going to be driving at three tests, one with Blender, creating a game, and creating a remotion video. We're going to be putting these two models head-to-head because as of now, you just have one more day left inside of your subscription with Cloud Code to use Fable 5. So, we need to see if it's actually worth paying the extra money if they decide to go usage-based, basically. So, right now you're subsidized, you're able to be able to get these access to these models to be able to actually use them inside of your subscription. And if Anthropic does nothing, then everyone is going to switch to OpenAI because Soul has actually been blowing me away for not just the quality, but for the cost that

0:45it actually takes to produce a quality output. So, we're going to be running those three tests today, and you can see here on Cursor Bench that basically Fable 5 high costs per task $8.77. And you can see Soul GPT 5.6 high costs $2.79. That is literally about 60-70% reduction in cost per task, which is absolutely insane. So, that's why we're going to be breaking it down. I also wanted to bring up, too, another one on design arena. For some reason, GPT models have always lacked taste, always lacked proper design. All the UI that I've done for my landing pages has all been with Opus, Sonnet, or Fable, and it's done an amazing job. But I think that Swift Soul

1:315.6 has pulled ahead. We're going to see how it does on UI design. We did do a test yesterday where Soul had actually replicated my landing page, and it did a good job. But you could see here it pulled ahead Fable 5, Opus 4.6, Opus 4.7, and then Grok coming in about 6-7 place. But GLM is kicking butt, GLM 5.2 on the design aspect. If you're using GLM 5.2, congratulations because it also is very effective on token consumption and it does not cost very much to run. Whether you're on the API or you have eight DGX sparks running in parallel, you could also run that model locally as well. So, the main goal we're going to see if Soul is actually

2:16worth switching to. That's the whole conclusion that we're going to come to today because basically you have 24 hours left to use Fable 5 inside of your subscription and we need to know. So, we're going to be pulling up these tests and getting them going. And we're going to be running it all of these benchmark tests inside of back-end, basically an ADE that I've designed to benchmark some of these model drops effectively. So, we rank them and then we have a panel of judges that are unbiased and that will judge the outputs and then we will post the findings on back-end bench. So, basically we're going to be able to go through three different tests today. Like I said, remotion video, Blender, and designing a video game. All of these

3:01models that we're going to be testing today, Fable and Soul, are going to be set to high effort. And then we're going to track the token consumption and how much it costs per output. So, the arena is staged and it's all logged in the rules markdown file. We're going to start with the horror horror game first and we're going to just see how it does. And then we're going to go into the Blender app and open it with the MCP add-on and we'll have it create what is called a donut test, which is a basic test for most people who do graphic design and see if these models can actually do it. And then we're going to go ahead and then do the remotion video, which is basically just going to be a redesign of my landing page to see how it does creating a remotion video. So, let's start it off. Hey back-end, tell

3:46Cedar to go ahead and start the testing for the horror game. >> Sent to Cedar. >> I'm going to set the Fable 5 to auto mode, both set on high effort, you can see here, and then high effort on GPT 5.6 Soul. And this is going to kick off the entire flow. In the prompt, are the same. The same prompt goes to Fable, the same prompt goes to Soul. It says, "Build a playable haunted horror horror game in a single self-contained HTML file, no external assets, libraries, or network requests. We could build it out in 3 JS as well. Maybe that's something we do down the road, but we're going to go with output here. What does it do with the tools that it has? The player explores a dark mansion, something is hunting them, and there is a way to win and a way to die. Prioritize atmosphere,

4:33darkness, lighting, sound, web audio API, and tension. Include a brief on-screen instructions. It must work simply by opening the file in the browser." Let's see how long this takes and how much tokens it consumes. Okay, this is very interesting, and this is what I noticed in my testing yesterday, is that Soul is done. You can see here, it says built. It includes first-person mansion exploration, procedural darkness, flashlight falloff, flickering, and film grain, web audio ambiance, footsteps, heartbeat, scare effects. I mean, it took 3 minutes, and you can see here, Fable 5 on high is still going at 5 minutes in. And I'd be interested in how many tokens it's consuming as well. But that should be the sign right here that not only is

5:19Soul on high that much faster, but it's using a ton of less resources to create a good quality output. But we need to compare the two just to see how they stack up. Because like I've always said, my time is very valuable, and if I can save more time by spending a little bit more money, you know, I don't mind because I don't want to have to go back and correct issues, correct bugs, and mistakes that Soul had possibly overlooked. So, if I'm using Fable 5 on medium, high, extra high, max, and it be it does the job perfect the first time, then that's going to be a cost worth considering. And I'm having basically Cedar, one of my agents in this workspace, basically manage the entire testing. So, you can see test one is rolling right here. Uh it's a cute

6:05delivery, landed clean. The first data point for the new configuration is watching how fast these models come back compared to yesterday's testing. We we ran a couple tests yesterday, but they ran so long because we were using ultra for Soul and we're using max reasoning for Fable 5. And the testing took hours for simple things. So, I think using ultra and max is just unnecessary unless you're running like a loop overnight or doing something letting them run autonomously while you go out and do other things. But sitting here and waiting, you're not going to need ultra or max for a lot of things. And especially if they're spinning up sub agents, be burning through tokens. So, I'm going to tell Cedar just to give us

6:50an update. Hey Becca, can you ask Cedar what the update is? I see that the Soul model had finished its testing for the game and Fable is still going. Just give me a quick update if you can. >> Right. I'll get a quick status from Cedar and then share the update. Sent to Cedar. >> All right. So, another cool thing about Hey Becca is that I can control everything by voice. And there is a way that I can also get announcements when a agent has finished its turn. It will then summarize it and announce agent completion out loud. So, I can turn this off or on. I think for some people it can maybe be a little bit annoying if you're running 12, 18, 25 agents. Um so,

7:35that's why we gave you the option to turn it on or off. So, let's see if it comes back and just gives us a quick update and then we'll go from there. But, boy, Fable 5 is taking longer than I thought. Can you check what Cedar was saying about the test results? Give me a quick summary. >> Let me check Cedar's recent output and pull out the key points for you. Cedar reports Souls run is complete and the build looks sound with both win and death paths working and basic checks passing. Fable is still running and hasn't produced an output yet. Once Fable finishes, Cedar plans a side-by-side playthrough, then comparison videos, and we'll alert you as soon as it's done. >> Sweet. So, yeah, like I said, we are posting basically everything on our website here. Anytime a model drops now,

8:21we're going to be posting here on Back and Bench and just giving people an idea of like the test results. So, going through actual like production code bases, fixing security vulnerabilities, refactoring, tests that we actually use day-to-day in real life. So, not only am I recording these testing videos for you, people can go and check out these test results. And there's going to be a judgment of three different models that also judge the output. So, there's an unbiased judge. So, here you could see Gemini 3.1 Pro voted, Grok 4.5 and GLM 5.2 voted. All this voting is done through Open Router, and it costs like I think yesterday it cost like $7 to do

9:06basically all the testing and voting. So, basically for dirt cheap, you could get a good analysis. So, you can see with the SVG self-portrait, Fable 5 Banks was the winner according to these three models. It says, "A model B perfectly captures the essence of a language model with its brilliant conceptual design using text paths, typography to represent the flow of the prompts and tokens." But, remember, if you didn't see my post, you should go check it out. But, I had just posted the cost breakdown. Basically, the cost breakdown yesterday from the testing is that Claude Code Fable 5 and Opus 4.8 cost 900 to 1,000 dollars in API pricing. 4.2 million tokens for 824

9:53millions were cashed in over 60 sessions. But, Codex GPT 5.6 running the same test was $50, 43 million cashed, 23 sessions, and 3.6 million tokens. So, the cash reads are the workload here, and I just thought that that was some interesting insight as to wow. Um for the quality, I mean, if you check this out here, I thought the Soul Ultra was better. It was animated. Fable 5 Max, it took like Okay, it took 17 minutes at 190,000 output tokens. Soul Ultra took 3 minutes at 10,000 output tokens. So, it literally cost like $5 to $10 running this test. Fable 5 Max, 10 bucks. Soul Ultra, 3 to 5 bucks. So, you got to

10:39decide on what is going to give you the best quality output because that is kind of ridiculous. And you could see here, Fable 5 is still going with the horror game. All right, looks like the benchmark is done. Can you tell Cedar to run the rule, and then I can do the comparison and check it out. Right. >> I'll pass that along to Cedar now. Sent to Cedar. >> All right, so it took 19 minutes here. You can see 19 minutes. It took It built and verified the haunted horror house game, and we'll be able to check it out. I want to see the comparison and go from there. Okay, the first test is going to be the one that Soul did, and you can see here the house that listens. It has been waiting for you. Three binding seals were left inside the Blackthorn

11:25house. Find them, bring them to the front door, and do not let what lives here to you. So, enter the house and let's see how this plays. All right. So, we have Souls game here and I honestly can't even see very well. I don't think it performed well at all. Um nonetheless, the controls work and the graphics are terrible and I don't even know where I'm going, but wow, for being done in 3 minutes, you know, this is very boring and all I can say is the controls work and I can't Okay, it has seen you. I was found. Oh my goodness. Something wears your footsteps now. Hard to try again. The house that listens.

12:10Enter the house. WASD to move, look, run, shift, interact. Map is floor plan. Oh boy. I mean, I guess for like the most simplest game you can ever make in the world and it's done in 3 minutes, the controls work, but the graphics are just terrible. Like, jeez, this is rough. Okay. Oh, and running works. Okay, so the controls work, but nonetheless, I know it was a very basic prompt, but boy, this is a a rough one to to watch. All right. Well, let's take a look at Fable's result. This is Fable's result right here, the hollow house. No one has left since 1911. So, again, same controls. Running is loud. It listens. The torch beam shows it. Let's see how it did. Sealed fast. Six

12:57sigils remain. So, this is like a top-down POV. Oh, I just it just crashed right when I took my first step. Horrible. Horrible. Horrible. F minus. It broke. The game literally just froze. Wow. That took 19 minutes. Jeez. And here's kind of the results of what happened. So, you can see on the right-hand side the game broke. On the left-hand side, you can actually play it. 3 minutes versus 19 minutes. 10,000 output tokens versus 200,000 output tokens. So, it's what? 25 So, it's $10 per million tokens in and it's $50 per million tokens out. So, this literally cost about what? $10 to make this game that breaks. Fable 5, F- Soul, C. Wow.

13:44>> ASpvin needs you. The benchmark site has been rebuilt with a minimal redesign and is ready for you to review before deploying. >> Great. Now, can you tell Cedar that we're on to the next part of the test? The test from the Haunted Horror House game has been completed and we should probably run the judgement panel on the arena to get a judgement score to post on our website and then we can move on to the next test. >> All right. I'll pass this along to Cedar now. Sent to Cedar. >> All right. It looks like here that the final results from the game is now done by our arena panel. You could see that Soul Ultra lost and Fable 5 Max won, which is interesting because I gave it

14:30an absolute terrible score four out of 10. So, this is the first true human panel split on a scored test. So, season Soul 28, Fable 14. Fable scorecard carries the ruling disclosure. Soul notes the ruling number three effort change. So, that's very interesting because the Fable 5 game didn't work and we were also on high, not max. So, it means that corrected, but so, tell Cedar that the results were measured on high. So, Fable 5 was on high and Soul was on high and I gave it such low score on Fable 5 because the game didn't work. So, yeah. Anyways, can you pull up the remotion videos so I can take a look?

15:16Tell that to Cedar. >> All right. I'll pass that along and then bring up the remotion videos for you. Sent to Cedar, I've opened the remote >> Okay, so the remotion video had finished and Soul took 5 minutes and Fable 5 took 8 minutes. I'm going to pop these up just to see it does a random remotion video on a product called night jar, but it's going to basically if you don't know what remotion is or the remotion skill is, it allows you to make product demo videos programmatically. So basically you could set up this skill inside of cloud code or inside of codex or the new chat GPT app, whatever you want to call it. And basically it makes beautiful demo videos of your product or any type of social media post. I think

16:02it's a very helpful skill to get used to using. Is it perfect? No, but it does with some finesse and back and forth with your agent, um, it can create beautiful demo videos that would used to have to cost a thousands of dollars to pay for years ago. Okay, so here's the test results here. Soul Ultra, like I said, 5 minutes with 10,000 output tokens. We got 8 minutes at 85,000 output tokens. So literally almost 10 times the output tokens, which is incredible. Let's see the results here. This is just a remotion video made night jar pretend, but look how good it does. Which one do you think is better? Do you think that Soul Ultra did better or Fable 5? In Fable 5, we again, the

16:48labeling is wrong. We're on Soul high and Fable 5 high. So we're not on Ultra or max anymore. That was yesterday's test. So I think Soul did a great job for literally 90% of the cost. During these last 2 days, I just think that Soul is going to be the better bang for your buck for a lot of people. No one's going to be paying usage credit pricing for Fable. I'm sorry, they're not. On to the next test, which is Blender. Okay, now tell Cedar that he can go on with the next test, which is the blender test. >> Right. I'll pass that along to Cedar now. Sent to Cedar. >> So inside of Blender, when you hit N after you install the Blender MCP, you can connect to MCP server and you can

17:33now make new You've been able to do this in the past, but now these models are so good. You can now create 3D animations for graphic design, for games, or for model design, or for architecture, landscaping, whatever you want to use Blender for via MCP, so you can now have these agents create things, which is a whole new level that just should blow your mind. So we're going to connect to the MCP server and we're going to go back to our agent and we're going to say, "Tell Cedar I have connected the MCP and we're now ready to go." >> I'll pass that update along to Cedar. Sent to Cedar. >> Okay, so it's going to be Soul on High running the first go-around inside of Blender, and you can say right here with the exact prompt here. It says The

18:18prompt is set up lighting for a camera for an appetizing composition. Then render a final image of 1920 by 1080 and save it to the disk, which is a soft donut, glossy icing, ceramic plate, and wood, which is considered to be a right of passage test for people who use Blender. So Soul's donut run is live. You can see here with the little emoji, and it fired verbatim and watch your Blender viewport and we can check it out. And this is it, live, building it right in front of our eyes. We'll see how it does. This is pretty cool, honestly, like just to be able to see this happen right before your eyes is pretty interesting, and we're going to be posting the results live on backendbench.ai.

19:04That is going to be going live later today, where we're basically keeping track of every single test that we do on this channel, whether it's Kimmy K 2.8 that comes out, GLM 5.3, or the new Muse Spark from Meta. Oh my goodness, that just came out the other day. All of these models we're going to be passing through rigorous tense testing inside of back and bench, uploading the results, and then you can take a look, make a decision for yourself before you commit to any subscription plan, try them out via the API and open router, and be able to actually use the model that's going to be great for your use case. Maybe you're making games right now. no idea. Well, this can be a great test that you can look back at and go, "You know what? Soul, for the money that it took and cost to create a donut inside of

19:49Blender, I think I'm going to choose that over something like Fable 5." This is going to be good data for people that are building cool games, building architecture designs, building web apps, mobile apps, whatever you're building, I'm just trying to give you the raw real results without hyping things up too much, right? Because there's so much hype in this industry. We need to see real results. I love benchmark scores, you know, they're great to look at. They get me hyped up. It's like when a new graphics card comes out, you're like, "Oh my goodness, it scored 500 frames per second on the new Fable game. Oh my goodness, great." But, you know, but when you use it in real life, and you're like, "Wow, my CPU's the bottleneck." Maybe it's not a good choice to go with that graphics card. That's how I kind of compare it with these model drops, for some reason. I don't know why, but hey,

20:34I'm a I'm a PC builder at heart. I've built many PCs, so I kind of have been like closely examining these model drops, comparing them to like new computer hardware releases. It's like reminding me of like the early 2000s when you used to build computers, and the new graphics card would come out, and you'd see the benchmarks, and you're like, "Wow, that's amazing." Shout out to people who ran PC Mark back in the day, and that that was like the the default right of pack passage scoring tool for hardware. And this is very similar. This is what it's reminding me of. Just brings me back to the good old days, but really cool to see this kind of come to life. Is it fast? No, it's kind of slow. Um but we'll see how Fable

21:19does cuz Fable is up next and then we'll post the results on the website. Okay, so you can see that the work was done in 4 minutes. Completed and verified. Everything is now staged for Fable 5 to do its work. So, that is amazing. That happened a lot faster than I thought. And then we're going to see how Fable 5 does on high to then compare the results and just see how they do with this donut test. Again, same prompt coming in for Fable. It says, "You are inside of Blender MCP. Build the classic Blender scene, a delicious-looking donut, icing, sprinkles sitting on a plate with a wooden table. Model is a donut, it's icing, blah blah blah, right?" Okay, so it's going to basically pass the same prompt. And same thing, Fable's donut

22:05run is live, verbatim, the prompt fired, and it is going. So, we're going to see it do its thing here. I'm going to be interested to see how long it takes. I'm guessing it's going to take longer just because that's what we've been seeing on these results. And then yes, exactly, when Fable's render lands, the benchmark rule runs, both artifacts will be filled in the results, stats will be pulled from the logs, and it will post the results inside of Becca'n Bench. Awesome. You can see here that Fable is doing its thing, ready to go. This is just after a minute. It is building it live. Pretty interesting. Wow, this is really cool seeing this come alive. Like, I just think that this is really amazing for people who build assets for

22:50games or any kind of game production or development. I just think that this is a complete unlock. You can download Blender for free, connect via their MCP, and have agents programmatically create anything that has to do with graphic design. I I think it's awesome. Okay, so the results are in and they are promising. Look at that here. You got Soul on high. It's not on ultra, but 4 minutes, 10,000 output tokens. You could see it did okay, but the one from Fable is just beautiful. Like look at that. I mean, it did such a good job. So much better than Soul. So, I mean, it's not going to change my mind. I think if you're going to be using these models

23:35and if Fable 5 truly comes off of your subscription tomorrow, you know, you're going to be using Soul and there is rumors and speculation that they will take Fable 5 off and Opus 5 will be released on the subscription plan. So, if that does truly happen, if Fable 5 does come off of the subscription, then expect to see Opus released here in the next few days and then that will kind of, you know, cause mass hype again. We'll go through the cycle again and we'll be wondering is Opus 5 better than GPT 5.6? And it's just a constant cycle, which is why we need to be focused on testing and actual use cases and in production code bases because we're all building different things. So, we need to be able to verify the test results and not just look at the beautiful benchmarks. So, what are your thoughts? Do you think that you're going to

24:21continue to use Fable 5 if it comes off of your subscription then paying for those usage credits, $10 per million input, $50 per million output or will you just wait till Opus 5 comes out? What do you think? I don't know. I think in overall, I think Soul did a great job. It was fast. It was cheap and as far as back end and crushing bugs and shipping features, I think it did a good job. It also did a really good job with the UI design in our last video. So, for the price, I think a lot of people are just going to be choosing the price per token model that's the lowest, which is going to be GBT 5.6, so you can use it on medium, high, ultra, whenever you choose. What are your thoughts?

25:06Remember, check out Back-End Bench to see the results. I'll put the link in the video, back-endbench.ai, and you'll want to bookmark that because we're going to be releasing benchmarks every single week of the latest and greatest models. We'll see you in the next video. Have a blessed day, guys. Bye-bye.