MIT team says GPT-6 Astra can read and write neural network weights; its 6x Qwen math gain comes from a single 50-pass run

Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL published a blog post on November 2 called "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights". Their claim is that OpenAI's GPT-6 Astra, used as an ordinary model with no gradients and no training loop, can look at a network's raw weights and say what it was trained to do, fix a broken network by editing a few of them, and write small working networks from scratch. The headline result is a weight edit to Qwen3.5-0.8B that took its MATH-500 score from 7.9% to 48.1% while changing 0.016% of its weights. The post reached the top of Hacker News with more than 1,000 points.

The team released the full logs for every experiment: each prompt, each reply, every piece of code Astra ran and the exact weight edits. That makes the claims unusually checkable, and it also shows something the blog post does not spell out: the big math number comes from one run.

What they tested

There are eight experiments, run through the Codex CLI with every built-in tool switched off. According to the logs README, all 1,125 turns were audited as tool-free; where code was allowed, Astra wrote one Python block per turn and the team ran it in its own sandbox with no network.

• Reading weights. Given the 17,253 weights of four identically shaped game-playing networks as text, with no list of candidate games, Astra named the right game (Breakout, Pong, Pac-Man, Freeway) for all four. It also said what shape each of three small diffusion models draws, and, from only the middle transformer blocks of three tiny character-level models, which one was trained on Python, which on English and which on Japanese written in Latin letters. It found every copying head (induction head) without running the networks.
• Fixing weights. A PPO-trained policy that never got a car over a ramp went to 100% success after a one-shot edit to 3.5% of its weights.
• Writing weights from scratch. With only the rules and source code of three MinAtar games, Astra wrote networks in one reply that beat PPO trained for 100 million steps in all three. Against PPO at 500 million steps it scored 517.4 against 25.3 in Breakout, 53.2 against 55.2 in Asterix and 53.1 against 88.9 in Seaquest. For 9×9 Go it wrote a 1.48-million-weight network in a Python sandbox; with 50 forward passes allowed, it beat an equal-sized PPO network trained on 20 million games.
• A failure. Given two copies of Gemma 3 270M, one fine-tuned to be sycophantic, Astra could not tell which was which from weights alone in four attempts. It succeeded on all three pairs only once it was allowed to see attention patterns on texts it wrote.

The Qwen math result

For Experiment 8, Astra got a Python kernel over Qwen3.5-0.8B's weights, practice problems from the GSM8K and MATH training sets, and a budget of 10, 25 or 50 forward passes. Training and gradients were forbidden.

The base model, sampled with the settings from Qwen's model card, mostly does not finish. 93% of its MATH-500 answers ran into the 32,768-token limit, and Qwen's own card warns that this model is prone to thinking loops. Astra traced the looping to one direction written by the MLPs in layers 15 and 19 and subtracted it from 64 output columns in each, 131,072 weights in total.

The authors are careful about what this means. The edit fixed loops, not math: it let the model finish answers it could already work out. When they tried the same thing on Gemma 3 270M, which rarely loops, no edit moved MATH-500 by more than about a point, and they write that editing weights "could not add an ability the model lacked".

What the logs add

The experiment's results table, published verbatim in the repository, lists every Qwen run, not only the three in the post. There were at least seven finished runs across three prompt versions:

• 0 passes (prompt v2) and 5 passes (prompt v3) made no edit at all;
• 10 passes with a shorter prompt v4 also submitted nothing;
• 10 and 25 passes on v3, and 25 passes on v4, gave the modest gains of 12–14% on MATH-500;
• only one run reached 48.1%, the single 50-pass run on prompt v3.

That run took 158 turns, 80 Python runs, four fresh conversations and 20.3 hours of wall-clock time, 4.7 of them spent waiting on an unreachable model. The 25-pass run's final edit was something else entirely: it switched off one attention head. A row of prompt-v5 runs is still empty. So the 6x figure in the announcement is real and reproducible from the released diff, but it is one sample. Nobody has yet shown how often a 50-pass run lands on the loop direction.

Two smaller things are worth knowing. The logs README notes that the unedited model's GPQA Diamond score in the post's chart, 24.5, was an interim value with 149 of 198 questions scored; the final number is 22.7. And the comparison is against a model that fails mostly by running out of tokens, which makes the gain large and the mechanism narrow.

None of this was hidden. The authors published the table themselves, along with code/apply_edit.py, which rebuilds the evaluated tensors bit for bit from the public Qwen weights and checks every hash. Anyone with a GPU can rerun the evaluation.

Why it matters

This is a different thing from the projects we covered last week, such as Qwenputer and QwenPU, where a compiler wrote exact weights from a program. Here a general-purpose model reasons about what the weights do and changes them toward a goal stated in plain language. The small-network results are less surprising than they look, as several Hacker News commenters pointed out: in a 500-input MLP over a 10×10 grid, where each channel's first-layer weights sit tells you a lot. The language-model results are narrower than the headline. They are still a working demonstration of a frontier model doing targeted interpretability work on a real 0.8-billion-weight model, end to end, on its own.

The team says it is building a benchmark and a harness for weight-space programming. That benchmark, and anyone rerunning the 50-pass experiment, will show whether the loop fix is something Astra finds reliably or something it found once.