Testing Kimi K3 and It's Actually Insane

0:00So, they're saying the new Kimi K3 beats Opus 4.8 on every single benchmark. Well, it just dropped yesterday at 3:33 p.m. Pacific time. In Moonshot, this is their main frontier model. We had Kimi K2.6, 2.7, lackluster models. Not very good, but Kimi K3 Ultra is a large-scale open-weight multimodal reasoning model from Moonshot. It is suited for complex coding, knowledge work, and long-horizon agentic workflows. This is particularly strong at navigating large repositories using tools, debugging, and entering against images, logs, tests, and runtime feedback. And you could see it's priced accordingly. We got $3 in and $15 out

0:47with a 1 million token context window. And in this video, we're going to be running it through some benchmark results and seeing if it really lives up to the hype. Let's get right into it. We're going to be running all the test results inside of Hay Backen, and we're going to be having basically Fable 5 manage the competitors. So, we have Opus 4.8 on max reasoning level, and then we have Kimi K3 at max effort level. And the first test that we have running is creating a Minecraft game. So, the prompt is create a playable Minecraft-style voxel game as a single-contained HTML file. No libraries, no assets, no network requests. It's just a first-person control, basically just recreating

1:32Minecraft. Let's see how long it takes and how many tokens that come in, how many tokens go out, so we can see if Kimi K3 can be an Opus 4.8 replacement. Just going over our Backen Bench rules, I want to just be really clear for you. We have fresh contacts with every single result. So, after we do this Minecraft test, we're going to be jumping into a Blender test, connecting it to the Blender MCP, and having to the donut test, which is a basic graphic design asset that everyone needs to learn how to do. So, Fable 5 results for the donut test via Blender were amazing. So, that's going to be another test we do. And just telling you like how we actually go about scoring these models, you can see here we use the same

2:17harness. All models in a session are run through the same tool with default settings, record which harness in the score.json file, capture everything. So, before scoring, we save the raw output to the results and take screenshots of the rendered result or a screen recorded video if it's basically animated and record a wall clock time with token counts and cost if available. We use the rubric score below and then we run while the test is fresh. So, this is kind of our workflow, the protocol for actually running the tests. And then we have an arena where we are connected with Open Router and we bring in three agents. That could be Gemini, that could be GLM

3:025.2, it could be Fable. Whoever comes in and judges the output of the results of these competitors. So, Opus 4.8 on Max and Kimi K3 are going to get judged in the arena and all the results get posted into backinbench.com. And here's how they actually test. The scoring rubric is 10 points. So, it's zero through three points. Basically runs first try, broken or refused. Three is flawless on first open. Two is minor glitches. Three is still works. One is partially works and zero is it didn't work at all. So, polish, visual aesthetic quality, prompt adherence, did it deliver every named requirement, and then the wow factor,

3:47and did it go beyond the ask in a way that made it run better. So, that is how we're scoring this out of total of 80 points. Here's some of the tests that we do. There's eight tests that we really put these models through to just make sure that they actually adhere and if I feel comfortable unleashing them on a production code base. The first test is usually building a game, some type of game, any type of game, Flappy Bird, Haunted Horror House game, Minecraft, Grand Theft Auto, whatever, just recreate it with minimal no libraries, CDNs, fonts, network requests, and external dependency cap prompt adherence at zero for that run. Test number six is a promotion video. Test four is a SVG self-portrait. Test five is a UI

4:33recreation, so giving it some modern UI, basically my landing pages, seeing if they can actually recreate it. Test seven is a debug gauntlet. Test eight is usually building something out in Blender. So those are the things that we're running these models through. They're up and running, they're up and going. Like I said, Fable 5 is basically orchestrating the two tests and they're going right before your eyes. Hey Back-end, you can tell Cedar to go ahead and start the tests and also ask Cedar, while he's starting the test, to open up a markdown file that has the test outline between Kimmy and Opus if there is >> Got it. Passing that along to Cedar now. Then I'll confirm. Said to Cedar. >> So like I said before, I really try to

5:18test these new model drops because benchmark scores are amazing and all the hype around these new model drops is getting a little ridiculous. So I thought, why not just create something for myself that I can actually benchmark these to see if I can use some of these models maybe inside of a swarm or as another orchestrator agent to be able to handle specific tasks. Cuz right now, I think most of our daily driver is Fable 5 or Soul 5.6 Ultra. But at the same point, do we need those type of models to handle everything? Not necessarily. So, that's why I built to back in bench and it is live right now back in bench.com. You could just check it out every time we release a new model that comes out, we run it through these test results, we post them live here. It's free, bookmark it, check it out. That way you can get a good idea of whether

6:04these models are actually worth using. All right, yeah, you can create a run sheet and then open it up, but let's first fire off the test results so they can start working on the test and then open up the run sheet. So, tell that to Cedar. >> All right, I'll pass those instructions along to Cedar in order. Sent to Cedar. >> You can see the models are now fired off. We have Opus 4.8 on max reasoning and max effort level here and then we have Kimmy K3 going at it on max as well and 1 million token context window for Kimmy K3 and they're actually starting on the lava lamp test and then we're going to circle back with the Minecraft recreation and then finishing up with a blender donut test as well. So, all

6:51orchestrated by Cedar. Cedar is basically running on Fable 5 high. It creates the plan, the outline and then keeps the agents in check when they're done with the tests, they get killed and then we start a new two new terminals with two new agents for the next test. Okay, so we have the run sheet right here. This is basically the outline for the two contestants today. How they're going to be judged, what they're going to be working on, lava lamp, Minecraft clone, blender donut, how we're going to score it, how we're going to collect the artifacts and also judge on how much it cost, how long it took for these tests to run. And it's important that we test each contestant result outside of the repo so neither auto connects the repo's build OS, which is our MCP, otherwise it

7:38would know that it's being tested. We want to verify zero MCP on both for both browser tests, and both are running at max per the ruling. So, this is just important that none of the tests are diluted or contaminated in any way possible. And then after each test, we're going to collect artifacts, pull real-time output tokens from the session logs, and just kind of see how much would this task cost if we're using the API, which is good to know. And about 5 minutes in, you could see Opus is still going, foraging, and you could see Kimmy K3 is pumping it out. Seems like it's both probably equal. I'm guessing Opus is going to take a lot of time because it's using a max effort, and it's just

8:23created realistic lava lamp simulation of a self-contained HTML file, and then it's going to open in the browser when it's all done. So, the Kimmy K3 is pretty interesting as far as it is a 2.8 trillion parameter model, and I just had to look that up because GPT-5.6 Soul is a 2.2 trillion parameter model. And right here, right before your eyes, you see Kimmy K3 finishing in in about 15 minutes, while Opus is still thinking. So, that means Kimmy literally got everything done within 15 minutes. The rendering is a fixed timestamp, 128 hertz physics decoupled frame rate. What

9:08we're seeing so far in the early early benchmark results is that Kimmy K3 is really good at UI and graphic design. So, I'm excited. That's why we're doing the game. That's why we're doing the lava lamp to test the physics. And that is why we're doing Blender, because if Kimmy K3 can outproduce Opus, we are in a whole new level as far as open weight models. And, you know, GLM 5.2 came out a couple weeks ago, people lost their minds. If Kimmy K3 can at least outcompete with Anthropic's frontier models, this is going to be opening the door for people to not only double down on buying their own hardware, but also creates a competitive advantage between some of these Chinese models. I'm already a big fan just with how fast it

9:53finished the first test because Opus is still running. It hasn't even written code yet. Okay, oh my goodness, 20 minutes later we have it going on. Oh my goodness, wow. That took forever. Hey Becca, tell Cedar that I just saw the results finish up. Can we check them out? >> All right, I'll send that over to Cedar now. Sent to Cedar. >> And like I've said, here is the run sheet and we're going to score these myself on camera, but we're also going to be putting them inside of the arena so that other models can score based on the scoring rubric. Wow, Kimmy K3 failed miserably. You can see Opus right here took 20 minutes and 180,000 output

10:39tokens, but Kimmy rendered nothing via the Kimmy code CLI. So basically, Kimmy K3 just failed straight up. It's a great data point and it's a brand new model via the CLI and the artifact whiffs on the core requirement while Opus has molten wax clearly going up and down throughout the lava lamp. So the scores I would say Kimmy K3 scores zero across the board and Opus gets 3 3 2 and 1. And that's just why we do these tests because we need to know is this something that I'm actually going to use as a daily driver or as a possible sub agent that handles specific tasks. So far, it's looking a little lackluster

11:25for Kimmy K3, I'm sorry to say. But you can't judge it just off of one test. That's why we're doing multiple tests in this video to see if Kimmy K3 can make a a comeback. So, I pulled up the actual video file of the test results here, and you can actually see Kimmy K3 did a decent job. Though the clarity is not all that good, and the lava itself has terrible physics, where Opus on the right hand side is much more representative of an actual lava lamp. 15 minutes versus 20 minutes, $4.50 via API pricing on what API Opus 4.8 charges. I don't know. I think the results obviously go to Opus here. Still, Kimmy K3 did not do a good job. It looks blurry, fuzzy, still pretty

12:10lackluster. On to the next test. All right, so can you tell Cedar that we are ready for the next test? >> All right, let me pass that along to Cedar now. Sent to Cedar. >> Okay, you can tell Cedar to start the game, and we're ready to go. >> Right, I'll send that along to Cedar now. Sent to Cedar. >> The prompt is create a playable Minecraft style game as a single self-contained HTML file. We have Kimmy K3 in a new terminal session with no memory or context of what we're doing. So, that's prompt's going to be handed handed off to Kimmy K3 on max effort. And then we have Opus 4.8 on max effort in auto mode going through the same results. So, we're going to be testing this out, creating a Minecraft game.

12:55Let's see how good it does because early results are showing Kimmy K3 being really good at game design. Let's see if that holds true. And just to keep you up-to-date, we ran the test for 34 minutes, and actually Opus finished first. While Kimmy is still writing the code, like I said, this is a 34-minute test. We have used 10% of our token context window out of a million in this new fresh test. And I have to say that so far the Kimi K3 results have been lackluster, but we've only ran one test. So, I'm interested to see how it does in this game because it's supposed to be really good at game design. Okay, we have something very special to share

13:41with you because this is pretty impressive. We just got the results back with the Minecraft game. 18 minutes later, Kimi K3 absolutely knocks it out of the park and you can see here it only took 18 minutes. While Opus 4.8 did a good job, did you know, nothing to write home about. It did a good job. It was also on max effort. You can see here 37 minutes, 425,000 output tokens, which would cost about $11 in the API. So, while the results aren't like mind-blowing, you're able to see the power of Kimi K3 here at essentially half the cost because you have $3 per million in and you have $10 per million out where Opus is just about double that. So, you need to start figuring out

14:27how am I going to orchestrate my agents because if Opus 4.8 is literally double the cost and I can get the same amount of work done at the same level of quality with Kimi K3, you need to ask that question to yourself. Let's get on to the Blender test to see how it does with building actual assets for the >> Talk to Cedar. >> So, I just told Cedar, that's running on Fable 5, to now start the next test inside of Blender. I did score the results from Kimi and Opus. All the benchmark results that you can go vote for yourself, by the way. Go to backandbench.com and go look at the results. Start putting in votes cuz that does matter because you'll be able to see it on your end as far as like is this something worth quality? Is this something actually amazing? And we let the people

15:13decide. I know we have an arena where we score everything with AI agents, but it's also good to get some human-to-human feedback, right? So, we're going to go ahead and start this next test inside of Blender, and we'll report back with the results. These tests inside of Blender are really fun to do because the AI agents, GLM 5.2, Fable, Soul, now Kimi K3, are now able to create game assets for whatever game that you want to build via the Blender MCP. So, this is really cool to watch live as it builds, and we're going to go ahead and run this test and just see how it does. So, we're going to get back into Beacon, and we are going to auto mode, auto mode, make sure all the

15:58models are set, and let's see. Okay, first up it's going to be Kimi. You could tell Cedar to go ahead and let Ki- Kimi cook let Kimi do its testing inside of Blender, and we will watch. All right, I'll send that along so we can observe the run. Set to Cedar. And while that's running, I just wanted to take a second here to talk about our free Discord and School Community we just launched. So, you could check it out. I'll put it link down below, but we have over 160 members inside of our School Community, and I just launched the Discord because where School kind of lacks is that communication. So, being able to like talk with you guys in an actual chat or voice-like setting, we just launched the Discord. It's called

16:44the Shipyard, where you basically just connect with other people who are shipping with these models, building cool things, running cool tests, and we'll be able to have more communication rather than just inside of the School Community. The School Community is full of more course-like materials and our live calls and results and challenges. So, if you want to be able to like connect with people like you and I, or other people that are in the community, I'll put the link to it down below. They're free. Check it out. Now, back to the video. So, you can see here it's starting to actually run the animation inside of Blender, which I think is so cool. It blows my mind that this is able to happen in 2026. It is so freaking cool. Kimmy K3 seems like it's done. Can

17:30you tell that to cedar and render it and run the rule? >> Send to cedar. >> While it's rendering, let's check out the Kimmy K3 version of Minecraft. This took 17 minutes to run and let's just see how good it did and like I mean, I'm not like an intense Minecraft player. I have played with the kids a bit, but this is pretty cool. Like it did a really good job. The graphics, obviously Minecraft's not known for its graphics, but it did a really good job and I'm pretty impressed by just, you know, being able to sign up for the $30 a month plan for Kimmy Code and being able to just knock out a video game that you could build upon. Obviously, this is

18:15just the first version. Think about if you actually spent time building upon this game, adding in assets, whether it's in Blender or whatever. I think it did a pretty cool job. Um, like here you could stack the blocks. Uh, let's see what five does. I mean, not not amazing, but compared to Opus and that output that took 35 minutes, I'm liking this. I give this an A+. And here is Opus output for the Minecraft version of the game and I can't even click into play. Like seriously, I can't even click. So, like I can't even get into the game. So, obviously that's a first first world problem, but at the same time it's like, okay, well, I can get into this game pretty pretty easy, pretty pretty easy

19:00and going into this one, I can't even control anything. What is going on? The results that just came out blow my mind when it comes to arena.ai. This is about design and Kimmy K3 just came in at number one beating Fable 5 inside of Code Arena. This is for web development. This is overall rankings across all models on front-end web dev including tasks, agent and coding workflows, and require multi-step reasoning and tool use. You can see here Kimmi K3 came in at number one beating Fable 5. The thing I want to highlight here is the price. For a third of the cost, you're able to get a better output out of Kimmi K3 which is a Chinese model from Moonshot

19:46out convening a US frontier model Fable 5 which absolutely blows my mind. And now it makes me want to test one more in this video when it comes to UI design recreation. And we're going to run that after this Blender test finishes. This is continuing to unfold right before our eyes. Like I said, this model just released within the last 24 hours. And it's taking the world by storm. I'm excited. Let's check on this Blender test and see what's going on. I have no idea. It's rendering the output from Kimmi K3 and then we're going to be starting on Opus here as it's going to run through its Blender test and then we'll run through the UI design test I think is going to absolutely crush Opus

20:324.8. You could tell Cedar that Opus is all set to start the Blender test. >> Got it. One moment while I pass that along to Cedar. Sent to Cedar. >> All right, while Opus is running its test, we had the Kimmi donut turn out. I think it turned out pretty good compared to Fable 5. Fable 5, I could show you here. This was Fable 5's result. I think Fable 5 obviously did a lot better of a job, but this also took like 40 minutes to render. So, here's Kimmi K3. Obviously, there's like no shot shadows or um you know, Boca where it's like slightly blurry like an actual natural photograph. I don't know. Like the stable five did a really good job, but you have to take into consideration it

21:19was like $30 to generate this. I mean, come on. That's not efficient. And you're probably wondering what the heck am I even using to run these benchmark results? Well, I built Hay Beacon a voice controlled ADE where I now have multiple projects going at the same time. So, it's just a way for me to manage all the apps that I'm building, all the projects that I'm building. We have built-in connectors here via MCP. You can connect GitHub, Idea Browser, Linear, Notion, Sentry, Vercel, Contact 7, Deep Wiki, Hugging Face, 21st Dev, Playwright, Fire Qual, Superbase, any any available MCP, you can connect your project to it. Also, we have a built-in MCP where your agents can communicate with each other across workspaces. Also,

22:06have implemented, which is amazing, an multiplayer vibe coding game. So, basically, people can join your project, bring over their own keys through OAuth, and start building alongside you whether whatever project that person's working on in their own separate Git work tree. And now we can multiplayer vibe code together. So, say you're stuck inside of your project and you want a family friend or a friend that you know that's into coding, bring them, invite them into your server that you can host, and you can start knocking out problems together. It is all voice controlled via GPT real time. We also have local Parakeet 3.0, which is Nvidia's local voice or text-to-speech model. And all

22:53of this happens right inside of Beacon. So, it isn't a workspace for you to manage projects. Right now, I have what, seven different projects going, and that's how I've been able to kind of keep track of all of my like mini second brains. Did I mention you have a basic second brain tied to every single project that is queryable via open knowledge format. What open knowledge format is a way for you to format your files more effectively so you don't run into a ton of token spend. So, open knowledge format is based upon the ICM methodology where it's basically organizing your markdown files inside of folders that is queryable across

23:39sessions, easily accessible from other agents, and you're able to basically build out your second brain more effectively. And that's the best way I can describe it without going into too much detail. I also have a usage counter here per workspace so you can keep track of how much you're spending and if you know you're going to be paying through the API, it's good to keep track of costs. So, we have that as well. We also have a built-in workbench. The workbench allows for your agents to keep track of tasks in an efficient way. We also have a looping feature where if you wanted to run agents inside of a loop, you can also do that as well. And you have a builder agent, you have a reviewer agent, you can choose the difficulty and effort level for the model. We also have

24:25a swarm feature. If you wanted to run an agent swarm, you're able to do that as well and set up a goal and a task and allow that swarm to just go crazy on whatever you're trying to build. And I'd say if you're not building in a way where you can voice control your entire workspace, I don't know. This is 2026. This is the way I've been liking to build for the last month and I'll put the link down in the description down below. We have a founder pricing included so you could check it out. OSX only but Windows coming within the next week. All right, it looks like Opus has just finished. Can you tell Cedar to check in on it? >> All right, let me quickly handle that for you. Sent to Cedar. >> While we're waiting for the Blender results to be rendered, I am going to be

25:12testing the new design capabilities of Kimiko 3. This is what we currently have on Beck and Bench. I feel like it's way too cluttered, way too much going on. It's unorganized. I don't know where to look for test results. There's too much text, way too much going on. We want it cleaned up. So, I'm going to have Kimiko 3 redesign our entire landing page, so it's easy to navigate, simple. I want to come to the site and I want to know what were the best results and in what area, either in debugging or UI design or security or creating a game or creating something in Blender. I want it to be easy and it's way too cluttered. So, we're going to let that run in the background while these Blender results finish up and we're going to just put it

25:57to the test. Can you tell Cedar to go ahead and start the UI tests of recreating Beck and Bench. I want it to look way more organized. So, you may need to add that into the prompt. >> Let me pass that along with the extra detail you mentioned. Sent to Cedar. >> All right, so the donut test officially rendered out. You could see Opus here rendered in 31 minutes and 8 seconds, 329,000 output tokens for $8 equivalent. And you can see here Kimiko 3 took 8 minutes. So, pretty comparable result. I would say Opus 4.8 is slightly better in the design, the shadows, the shading, but Kimiko 3 for getting it done literally in 70% less time, I think it's going to

26:44go to Kimiko 3. I do notice though, Kimiko 3 does take a little bit longer than expected. I thought it was going to be a lot faster. I believe it's 27 tokens per second. And Opus 4.8 no notably is always slower when you're running on max effort. I never run Fable 5 or Opus 4.8 on max, but I thought to give this a good comparison as far as a test matching Kimmy K3 on max and Opus 4.0 max sounded fair. But what do you think? What are your results? What do you think? You could vote now on backandbench.com and vote your results and see what you think. So here's what we have on the redesign. You can see Kimmy K3's redesign for the original. Here is the original back and bench. Kind of things are a little bit

27:29unorganized. I don't really know where to look as far as a scoreboard. And you can see Kimmy K3 actually went through the entire site and replicated it beautifully. Yes, there's no motion frame or motion here, but it did a great job at organizing the actual scoreboard, the human scoreboard, the panel scoreboard, and the eight tests that we go through and as far as efficiency. It didn't do any gradients. It also didn't mimic the entire background like I had the original, but I think it did a really good job for how long it took in only eight minutes. So this has been a pretty interesting test to run through these four. I'm going to continue testing Kimmy K3 in the next few hours and uploading everything to backandbench. Please go on there to

28:15actually cast your vote, see how well it does because hey, Kimmy K3 front end design number one beats Fable 5. And I wanted to show you here on Artificial Analysis, the results just came out that Kimmy K3 officially on the Intelligence Index beats Opus 4.8 on max. So not only is it showing up in real world production tasks, you're seeing it show up everywhere on benchmarks. I want to know from you, how are you using Kimmy K3? Do you think Anthropic and OpenAI have lost their edge now when it comes to these model drops? Seems like they're becoming less and less more appealing and with the drop of GLM 5.2 coming out the last few weeks and now Kimiko 3, these open weight models are getting significantly better versus frontier

29:02model drops. Like I said, check out the results down below, join the free Discord, the free community, and comment down below, what are your thoughts on Kimiko 3? We'll see you in the next one, guys. Have a blessed day. Bye-bye.