QwenPU puts Karpathy's stories260K inside Qwen 3.5, running on a vector unit compiled into its weights

Divyansh Agrawal, the developer who compiled a RISC-V CPU into Qwen 3.5's weights, has now put a second language model inside the first one. QwenPU, published on October 31, is Qwen3.5-4B with that CPU plus a new vector unit compiled into its weights, and it runs Andrej Karpathy's stories260K, the 260,000-parameter Llama from llama2.c, as a C program inside its own forward passes. The tiny model writes its children's story token by token, and every step of the arithmetic behind it happens in layers that a compiler wrote rather than training.

We covered Qwenputer, the original CPU-in-the-weights project, on October 29, and a day later nesformer, which used the same technique to run a Nintendo's 6502. QwenPU is the first follow-up from Agrawal himself, and it answers the obvious question those two left open: can this kind of in-weights computer do the one workload a language model is actually built for?

Why a CPU alone wasn't enough

Running a neural network on Qwenputer's plain RV32IM core is possible, just absurd. According to the README, stories260K compiled as ordinary C with int8 weights takes about 2.43 million trace tokens of CPU execution for each token of story. That is about 46 times the work QwenPU now needs per token, and the reason a dedicated unit was worth building.

Qwenputer already had an experimental "NPU" that gave the CPU dot-product instructions, which cut that to 225,000 trace tokens per story token. QwenPU replaces it with a dedicated vector unit that has its own memories and streams whole results through the token stream, so a 64-lane matrix row costs a few tokens instead of dozens of instructions. The count drops to 53,000 trace tokens per story token, and each story token takes about 16 seconds, against 41 seconds for the Qwenputer NPU. All three builds produce the same story, word for word: "Once upon a time, there was a little girl named Lily..."

What the vector unit holds

As in Qwenputer, the memory lives in the model's Gated DeltaNet layers, the linear-recurrent heads that make up most of Qwen 3.5's attention stack. With the gate held open and one-hot keys, each write overwrites exactly one row of a head's state, so the state behaves like addressable RAM. QwenPU puts the unit's memories in DeltaNet heads in the computer's first layers: 16 vector registers of 64 words, 384 int8 activation lanes, a weight memory of 4,152 row-chunks of 64 int4 weights, and K and V caches for 32 positions.

The instruction set reads like a minimal inference accelerator. VMAT does int8 by int4 matrix rows, VMATR computes attention scores over the K cache and multiplies probabilities by the V cache, VEXP produces softmax numerators from an exponent table, VEW handles residual adds, the SiLU-gated product and RoPE, and VQ quantizes results back to int8. SiLU and exp are fixed-point function units built into the circuit, not learned approximations. The design notes are frank about how tight the fit is: the CPU's datapaths already fill layers 5 to 9, so the vector unit had to be squeezed into the early layers' spare attention heads and MLP units.

One design choice matters more than it looks. stories260K's int4 weights are not compiled into the checkpoint; they are data, loaded into the unit's weight memory by the program. The README notes that any model that fits would run the same way. What was compiled is a general, if very small, accelerator, not a hard-wired copy of one network.

How exact, and how big

The project's checks are thorough. Every vector instruction runs random test programs teacher-forced against a reference implementation, and the whole story program, all 3,184,920 steps, ran free on the GPU executor and matched the reference at every step. The merged checkpoint's own layers were checked on the test programs and the first 30,000 steps of the story.

The cost is size. The computer now takes 13 compiled layers in front of Qwen's 32 (Qwenputer used 11), the hidden size grows from Qwen's 2,560 to 5,120, and the checkpoint is 26.7 GB in bf16, up from Qwenputer's 18.8 GB and, by the author's account, mostly zero padding. Loading it whole needs a GPU with 40 GB or more; the story demo runs the computer's layers separately and fits in 24 GB. Qwen's own weights are unchanged. Agrawal marks the project as a work in progress and says trimming the hidden size, and with it the checkpoint, comes next, along with a full write-up he hasn't published yet.

Divide the step count by the per-token figure and the story is about 60 tokens long, which at 16 seconds a token comes to roughly a quarter of an hour on an L4 (our arithmetic from the README's numbers). Nobody should mistake this for a way to serve models. The more useful reading is the one Qwenputer suggested: the recurrent half of a hybrid model can hold exact, structured state, enough to run a CPU, a vector unit, attention and a KV cache of its own, while Qwen's own weights stay exactly as they were.