Weight-Space Programming Explained: GPT-6 Astra Edits Qwen3.5

By Beau Johnson··9 min read

Quick answer

MIT CSAIL researchers had OpenAI's GPT-6 Astra edit Qwen3.5-0.8B's weights in a Python notebook with no training. It found the direction behind the model's thinking loops, changed 131,072 weights (0.016%), and took MATH-500 from 7.9% to 48.1%. It fixed a reliability problem. It didn't teach the model new skills.

In this video I break down how OpenAI's GPT-6 Astra opened up another AI model, changed 0.016% of its weights, and took its MATH-500 score from 7.9% to 48.1% without training it at all. If you build with AI and want to know what "weight-space programming" actually means, how Astra did it step by step, and how much of the hype holds up, this one is for you.

What I cover in this video

  • 0:00 GPT-6 Astra edited another AI's weights
  • 0:46 What happened: the looping Qwen model and the MIT CSAIL setup
  • 3:01 How it works, step by step
  • 6:04 The results on MATH-500, GSM8K, IFEval and GPQA Diamond
  • 7:35 The receipts and the critics
  • 9:54 The limit: why it couldn't fix Google's Gemma
  • 11:25 The weirder experiments: naming games from raw weights and writing networks from scratch
  • 12:11 What you can try today as a builder

What actually happened

The patient was a tiny open model, Qwen3.5-0.8B. It had a habit of getting stuck: it would ask itself the same question, answer it, then ask it again. 93% of its answers on MATH-500 ran out of room before they finished. Qwen's own model card even warns that it's more prone to thinking loops.

On November 2nd, four researchers at MIT CSAIL, including Rikiya Takehi and Omar Khattab, published a blog post called Sparks of Weight-Space Programming. It's a blog post, not a peer-reviewed paper, and they call these very early results. They handed Qwen's weights to GPT-6 Astra and told it to make the model better at math.

Astra ran through OpenAI's Codex CLI with its built-in tools switched off and reasoning turned all the way up. It got a Python notebook on top of Qwen's weights with no GPU and no internet, almost 15,000 practice math problems, and a budget of 10, 25, or 50 forward passes. A forward pass just means running the model: you feed it problems, it answers, and you can watch the numbers moving inside it. Nothing in the weights changes. Training was banned, so every weight change had to be worked out and written by Astra itself, and it never saw the test. The 50-pass run took 158 turns, 80 Python runs, and about 20 hours, with 4.7 of those hours spent with the model unreachable.

How Astra fixed the model, step by step

A quick mental model first. As Qwen reads each word, it keeps a running list of 1,024 numbers, like a shared notepad. Each of its 24 layers reads that notepad and adds notes. A direction is a pattern across those 1,024 numbers, basically an arrow, and some behaviors line up with one arrow. Researchers showed back in 2024 that refusal in chat models lines up with a single direction like this, so the idea isn't new. Inside each layer are 3,584 neurons, and when a neuron fires it writes a fixed pattern onto the notepad. That pattern is one column in a weight matrix called down_proj.

Here's what Astra did, straight from the released logs:

  1. Diagnose. It ran the original model on 96 practice problems. 11 were right, 85 hit the token limit, and every answer that actually finished was correct. The model wasn't bad at math. It couldn't stop talking.
  2. Build the arrow. It wrote pairs of text: the same problem continued normally and continued in a loop (one of its loops: "But is 20 the total? No, it is the remaining time. But is 20 the total?"). The difference in the notepad between the two is the loop direction.
  3. Start small. Switching off one neuron gave about six points on 32 problems. Astra itself said that wasn't conclusive.
  4. Go wider. It subtracted the loop arrow from what layers 15 and 19 write. On fresh problems, correct answers went from 11 of 128 to 42.
  5. Turn the dial. Strengths one, two, and three kept increasing the gains.
  6. Run a control. This is my favorite part. An edit of the same size along a random arrow did basically nothing, so this specific arrow is what matters.
  7. Sharpen it. It picked the 64 neurons in each layer that write the loop arrow the most, took out their loop component, and kept going to strength 6, so they now push slightly against looping. 64 columns x 1,024 numbers x 2 layers = 131,072 weights.

On 1,152 practice problems it never used to pick the edit, accuracy went from 10.6% to 60.4%, and the typical correct answer was still over 9,000 tokens long. So it didn't make the model quit thinking early. It let it finish.

The results, the receipts, and the critics

On the real tests: MATH-500 went from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, IFEval (instruction following) from 21.0% to 62.3%, and GPQA Diamond from 22.7% to 36.6%. A math fix helped instruction following because looping was never a math problem. It was a finishing problem.

The budget mattered a lot. With 10 passes MATH-500 only reached 12.3%, and with 25 it reached 14.1%, and each run ended on a different fix. Only the 50-pass run had room to test strengths, run a control, and land on the targeted edit. And loops didn't disappear: 33.1% of answers still hit the limit, down from 93%.

I give the team a lot of credit for the receipts. They published the full logs for every run: every prompt, reply, line of code, and forward pass. Only two weight matrices in the edited model differ from the public Qwen. Ahsen Khaliq at Hugging Face, working with Hugging Face's ML Intern agent, rebuilt the edit from the logs and got 51.0% on MATH-500 with 31.6% hitting the limit.

The second opinions are mostly that this is less magic than it looks. Neel Nanda said Astra is "surprisingly capable of doing old school mechanistic interpretability research rather than anything spooky." Matthew Honnibal pointed out that the summary made it sound like the result came from just reading the raw weights, which it didn't. Astra's task prompt also included a hint: "if the model often fails because it repeats itself and never finishes, finding and fixing what drives that repetition counts." Co-author Alex Jiang said the team didn't realize it until after they got the result. The authors themselves called the reaction overhyped, and on November 3rd they updated their summary to mention tools and algorithms. TurinTech also found a cost: tool calling dropped from 292 of 400 to 261 of 400 after the edit.

My take: the critics are right that the technique isn't new. The new part is a general model choosing that technique, running it, checking it against a control, and shipping a working fix from a notebook.

The limit, and what you can try today

On Google's Gemma 3 270M, Astra couldn't help. Gemma started at 6.8% on MATH-500, only 8.5% of its answers hit its token limit, and no edit moved it more than about a point. Its mistakes were ordinary ones like wrong arithmetic. As the authors put it, editing weights could remove an obstacle that hid an ability, but it couldn't add an ability the model lacked.

The rest of the post gets stranger. Astra named which game four small networks were trained on (Breakout, Pong, Pac-Man, Freeway) from raw weights as plain text, and it wrote a Breakout network from scratch that scored 517.4, versus 15.8 for a PPO agent at 100 million steps and 25.3 at 500 million. It did lose to PPO on Seaquest, though.

Three things you can do now:

  • Try the demo Space to compare the original and edited models side by side. The live tab caps answers short, so check the tab with recorded full-length runs.
  • Download the edited model. At 0.8B it runs on basically any laptop.
  • Read the logs. Astra's process is a recipe you can reuse with your own AI agents: diagnose first, find the cause, test at different strengths, then check on problems you didn't use.

Key takeaways

  • GPT-6 Astra changed 131,072 of Qwen3.5-0.8B's weights with no training and took MATH-500 from 7.9% to 48.1%.
  • The fix targeted a looping direction in layers 15 and 19, confirmed with a random-arrow control.
  • It fixed reliability, not capability. On Gemma, which rarely loops, it couldn't add skills.
  • The logs are public and Hugging Face reproduced the result, but the prompt contained a hint and tool calling got worse.
  • I wouldn't put this near a production model yet: one run per budget, about 20 hours of frontier model time, and the authors themselves call it very early results.

FAQ

What is weight-space programming?

In the MIT CSAIL experiments, it means an AI model directly reading or writing another model's weights instead of training it. In the Qwen case, GPT-6 Astra worked out an edit in a Python notebook and wrote it into the weights itself.

How did GPT-6 Astra improve Qwen3.5-0.8B at math?

It found that the model mostly failed because it got stuck in thinking loops, built a loop direction from paired normal and looping text, and removed that direction from 64 neurons in each of layers 15 and 19. That let the model finish more answers, and MATH-500 went from 7.9% to 48.1%.

Did Astra teach Qwen new math skills?

No. Qwen could already solve many of the problems but ran out of room before finishing. On Gemma 3 270M, whose mistakes were ordinary ones, no edit moved the score more than about a point.

Has anyone reproduced the result?

Yes. Ahsen Khaliq at Hugging Face rebuilt the edit from the released logs and got 51.0% on MATH-500. TurinTech's outside test found tool calling dropped from 292 of 400 to 261 of 400 after the edit.

Chapters

  1. 0:00GPT-6 Astra edited another AI's weights
  2. 0:41What happened
  3. 2:33Join Shipping Skool
  4. 2:52How it works, step by step
  5. 6:04The results
  6. 7:39The receipts and the critics
  7. 10:16The limit: Gemma
  8. 11:19Where it gets weirder
  9. 12:36What you can try today
  10. 13:54Would you let an AI edit your model?

Transcript

Read the full transcript

0:00OpenAI's GPT6 Astra just went inside another AI's brain, changed 0.016% of its weights, and took its score on a math test called Math 500 from 7.9% to 48.1. And it wasn't allowed to train it at all. The post is at nearly 10 million views as of November 5th, and some of the top interpretability researchers in the world are saying it's a lot less magic than it looks. So, in this video, we're going to be breaking down exactly what Astra did step by step, the results, the receipts, what it couldn't fix, and what you can actually try today as a builder. Let's jump right into it. So, here's what actually happened. The patient was a tiny open model called

0:46Quen 3.5.8B. And this model had a problem. It would start working on a math problem, ask itself the same question, answer it, and then ask it again and again. 93% of its answers on math 500 ran out of room before they ever finished. Even Quen's own model card warns that it's more prone to getting stuck in thinking loops. So on November 2nd, four researchers at MIT CSIL, Rakia Teahi, Jaden Teao, Alex Jiang, and Omar Katab put out a blog post called Sparks of Weight Space Programming. It's a blog post, not a peer-reviewed paper, and they call these very early results. They handed Quen's Weights to OpenAI's GPT6 Astra and said, "Make it better at

1:31math." They ran Astra through OpenAI's codec CLI with its built-in tools switched off and its reasoning turned all the way up. And the operating room was pretty bare. A Python notebook sitting on top of Quen's weights with no GPU and no internet. Almost 15,000 practice math problems and a budget of forward passes, either 10, 25, or 50. So what's a forward pass? It's just running the model. You feed it a batch of problems, it answers them, and you can look at the numbers moving around inside it while it works. Nothing in the weights changes. Training is where an optimizer nudges every single weight over and over automatically. Astra was flatout banned from doing that. It could run the model to watch it, but every change to the weights, it had to work

2:16out and write itself, and it never saw the test. On the 50 pass run, the whole thing took 158 turns, 80 Python runs, and about 20 hours, and almost five of those hours were just the model not responding. And I thought Slopus was slow. Real quick, 263 builders are already inside Shipping School. In Premium, I show you how I make every video on this channel with Claude Code. New course material drops every single day, plus live calls every week, over 500 hours of replays, and a full course library. Join free link below. So, how do you fix a model you're not even allowed to train? Don't get lost in the sauce here. It's simpler than it sounds. So, a model's weights are just numbers,

3:01a long list of them. Quen has about 0.8 billion. Now, as the model reads each word, it keeps a running list of 1,024 numbers for it. Think of it like a shared notepad. Quen has 24 layers and each layer reads that notepad and adds its own notes. A direction is a pattern across those 1,024 numbers. Basically, an arrow pointing one way. And some behaviors line up with one arrow. So, the more the notepad points that way, the more the model does that thing. Researchers showed back in 2024 that refusal in chat models lines up with a single direction like this. So, that idea isn't new. Remember that because it comes back later. Inside each layer, there's a block of 3,584 little detectors called neurons. When a

3:48neuron fires, it writes a fixed pattern of 1,024 numbers onto that notepad. And that pattern is one column in a weight matrix called downproj. So here's how Astra actually did it straight from the released logs. Step one, diagnose. Astra ran the original model on 96 practice problems. 11 came back right and 85 ran all the way to the token limit and every answer that actually finished was correct. So the model wasn't bad at math. It just couldn't stop talking. So step two, build the arrow. Astra wrote out pairs of text. The same problem continued normally and the same problem continued in a loop. Here's one of the loops it wrote. But is 20 the total? No, it is the remaining time. But is 20 the total? So then it ran both versions

4:34through the model and took the difference in the notepad between them. That difference is the loop direction. Step three, start small. It switched off one single neuron and it got about six points better on 32 problems. Astra itself said that wasn't conclusive. So step four, go wider. It subtracted the loop arrow from what layers 15 and 19 right. On a fresh set of problems, the model went from 11 out of 128 right to 42. Step five, turn the dial. It tried strength one, then two, then three, and the gains kept growing every time. So, step six is the control, and this is my favorite part. It made an edit the exact same size, but along a random arrow. Basically, nothing happened. So, it

5:19really is this arrow that matters. And step seven, sharpen it. Astra picked the 64 neurons in each of those two layers that write the loop arrow the most. In each one, it took out the part pointing along the loop arrow and then kept going at a strength of six. So those neurons now push slightly against looping. 64 columns times 1,024 numbers times two layers. That's your 131,072. So on 1,152 practice problems, it never used to pick the edit. The model went from 10.6% right to 60.4. And the typical correct answer was still over 9,000 tokens long. So Astra didn't just make the model quit thinking early. It let it finish. So

6:04what did that do on the real tests? On math 500, the model went from 7.9% to 48.1. That's about six times higher. On GSM 8K, which is grade school math word problems, 13.8 to 67.1. And on EPA valal, which checks if a model follows instructions, 21 to 62.3. So instruction following roughly tripled from a math fix, which is insane. And it shows up on harder tests, too. GPQA Diamond went from 22.7 to 36.6. So why would a math fix help a model follow instructions? Because looping was never a math problem. It was a finishing problem. get the model unstuck and it

6:50finishes a lot more of everything. It's still the same model underneath. It just gets stuck a lot less often before the end. Now, here's what this is all about. The budget mattered a lot. With 10 forward passes, math 500 only got to 12.3 with 25 14.1. And each of those three runs ended on a totally different fix. The 10 pass run took a loop direction out of 30 different matrices. The 25 pass run switched off one attention head. Only the 50 pass run had enough room to test strengths and run a control and land on the targeted one. And the loops didn't disappear. 33.1% of answers still hit the limit, down from 93. So about 1 in three answers still

7:35runs out of room. Much better, but it's not fixed. So is any of this real? This is where I give them a lot of credit. They put out the full logs for every run in the post. Every prompt, every reply, every line of code Astra wrote, every forward pass. It's all on GitHub. And that repo was at about a thousand stars when I checked on November 5th. And in the edited model, only two weight matrices differ from the public Quinn in layers 15 and 19. And then Ascen Khalik at HuggingFace working with HuggingFace's own AI agent rebuilt the edit from those logs and reran the full math 500 test on the edited model. It scored 51% with 31.6% of answers hitting

8:20the limit. That's a little better than the blog's own number from outside the team using the same setup. But now the second opinions and there are a lot of them. Adam Carvin put it this way. Astra used code to perform some standard interp techniques, found a looping direction, and abladed it from the weights. Neil Nandanda, who co-wrote that 2024 paper I mentioned, said Astra is surprisingly capable of doing old school mechanistic interpretability research rather than anything spooky. And the first author actually agreed with him and said experiment 8 used a similar idea. Matthew Honibal wrote that if you read the summary, you'd assume the Quen result worked just by looking at the raw weights. Well, it doesn't. And here's something most people don't think about. Astra's task prompt

9:06basically gave it the hint. It said, for example, if the model often fails because it repeats itself and never finishes, finding and fixing what drives that repetition counts. Co-author Alex Jiang said the prompt does include hints and that the team didn't even realize it until after they got the result. The authors themselves are pumping the brakes, too. Jaden Tio said the reception has been more inflated than they expected. Alex Cien called the public reaction extremely overhyped, but said overall it's a good idea with interesting findings. And on November 3rd, they changed the first line of their summary to mention tools and algorithms. And one outside test found a cost. A company called Turk tested tool calling, and the edited model got 261 out of 400, down from 292 for the

9:54original. So where does that leave us? I think the critics are right that the technique isn't new. What's new is a general model picking that technique, running it, checking it against a control, and shipping a working fix from a notebook. The prompt coached the process, but nobody told it the technique. That part is pretty impressive. The rest, take it for what you will. So, if Astra can fix Quen, can it fix any model? The researchers tried the same setup on Google's Gemma 3 270M. That's a smaller model, about a third the size of Quinn, and it rarely loops. Only 8.5% of its answers hit its token limit. And here, Astra could not improve it. Gemma started at 6.8% on Math 500,

10:39and none of the edits moved it by more than about a point. Gemma's mistakes were just ordinary mistakes, like wrong arithmetic, and there was no single part to switch off. So, the authors put it really well. Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked. Omar Katab said the same thing on X. This isn't about capability, just reliability. Quen could already work out a lot of these answers. It just couldn't stop talking long enough to give you the answer. Astra cut down the talking and it didn't teach it any math. Gemma's not a samesized test and it ran with different settings, but the point still stands. So, the Quen surgery got all the attention, but the rest of the post gets a lot weirder. In the first

11:25experiment, Astro got the raw weights of small game playing networks, 17,253 numbers each, as plain text with no tools at all, and it named which game each network was trained on, all four of them: Breakout, Pong, Pac-Man, and Freeway. Simon Willis then tried the same prompts with Quen 3.827B 827B on his laptop and it got three out of four in about an hour each on a laptop. So then it went from editing to writing. Astra wrote gameplay networks from scratch, weights and all in one reply. On Breakout, its network scored 517.4. A standard PO agent trained for 100 million steps scored 15.8. Even against

12:11PO at 500 million steps, it scored about 20 times as much. And the network it wrote basically does one thing. It works out where the ball is going to land and moves the paddle there. Come on now. To be fair, it lost to that bigger PO agent on Sequest and it was just short on asterisk. And one critic, fake SciHo, said the more you dig into each experiment, the less magical it gets. So what can you actually do with this today? Three things. So first, try the demo. It's a hugging face space where you can run the original model and the edited one side by side. The live tab caps answers short by default. So there's also a tab with the recorded fulllength runs. I'll put the link down below. Second, download the edited

12:56model. It's on Hugging Face. It's 0.8B, so it'll run on basically any laptop. Third, and this is the one I'd actually spend time on, read the logs. Astra's task prompt is basically a recipe you can steal. diagnose first, find the cause, test it at different strengths, and then check it on problems you didn't use, and now you can hand that whole playbook to a model. People are already trying it on other models, and none of that had been checked when I looked, but you can see where this is going. So, here's where I land. I wouldn't let this anywhere near a production model yet. It was one run per budget with a hint in the prompt about 20 hours of a frontier model for a fix a good interpretability researcher could make and one outside test already found a side effect. It

13:43patched a breakdown in Quen and it didn't add a skill. The authors themselves call these sparks and very early results. Omar Katab says evals for this are coming so stay tuned. So remember that loop but is 20 the total over and over. one a I went into another one with a notebook, found the arrow behind that loop, and turned it down. And now the model gets to the end of a lot more of its answers. Normally, you'd fix a model by training it some more. Here, one model patched another one without any training at all. And on Gemma, it couldn't add a skill that wasn't there. So, would you let an AI open up your model and change its weights, or would you rather just retrain it? I want to hear your take. So, let me know below. And if this made weight space a little less of a mystery,

14:29tap the thumbs up so more builders see it. Have a blessed day.

Ready to start building with AI?

Join the Shipping Skool community free. Paid members get live bootcamp calls every week and weekly-updated courses.

Join Shipping Skool

Keep reading