Qwenputer compiles a RISC-V CPU into Qwen 3.5's weights and uses its DeltaNet layers as RAM

Divyansh Agrawal, an independent developer, has taken a stock Qwen 3.5 4B and added a 32-bit RISC-V processor to it. Not as a tool, not as a sandbox the model calls out to, but as extra layers whose weights were set by a compiler rather than learned. The processor's memory lives in the model's own Gated DeltaNet layers. When Qwen needs to multiply two large numbers, the CPU runs inside the forward pass and the answer comes back as generated tokens.

The project, called Qwenputer, was published on October 22 with code on GitHub and weights on Hugging Face, and reached the Hacker News front page this week. It is a stunt in the best sense: the useful part is not the calculator, but what it shows about the recurrent layers that newer hybrid models are built on.

Recurrent state as addressable memory

Qwen 3.5, like Qwen3-Next before it, uses Gated DeltaNet, a linear RNN, in three of every four attention layers. Each head keeps a fixed-size state matrix. At every token the state is multiplied by a gate between 0 and 1 (that is how it forgets), and then one direction of it is pushed toward a new value with a write strength called beta.

Agrawal's observation is that if you set the gate to 1, beta to 1 and use one-hot keys, every write overwrites exactly one row of the state and every one-hot query reads back an exact value. The RNN stops being a fuzzy summary of the past and becomes RAM with addresses. Unlike a KV cache, it does not grow with the conversation.

The idea that delta-rule linear attention behaves like an associative memory is not new; Schlag, Irie and Schmidhuber made that argument in 2021 in "Linear Transformers Are Secretly Fast Weight Programmers", and the post cites it. What is new is making the memory exact inside a production checkpoint and leaning on it hard enough to run a processor.

A CPU written, not trained

The CPU is described as a circuit: read a register, add two values, jump if zero. A compiler turns each step into pieces of transformer layers, with attention heads for lookups, feed-forward units for arithmetic and DeltaNet heads for memory. The result is an RV32IM core with the Zba extension and a small bignum unit, occupying 11 layers placed in front of Qwen's 32. Qwen's original weights are untouched; the computer sits in added layers and extra hidden dimensions.

Execution is visible as tokens. Each instruction becomes a few of them: an address to write, a value, and a step marker. According to the author, the core passes the official riscv-tests, matches qemu on its programs, ran 18 million instructions free-running without an error, and replaying about 230,000 output positions against a reference machine produced no mismatches. It can run C and MicroPython, slowly.

Qwen talks to it through the tool-call format it already knows, so no fine-tuning was needed. A small driver, RvChat, loads the firmware into the DeltaNet state and keeps the trace tokens out of Qwen's own layers.

Does it beat a Python tool?

Mostly it matches one, which is the honest answer and probably the right expectation. On the author's runs over DeepMind's arithmetic sets, base Qwen 3.5 4B scores 72.5% on div_big and 55% on mul_big; Qwenputer scores 97.5% on both. Qwen with an ordinary Python tool scores 95% and 100%. On NUPA long-integer arithmetic, Qwenputer and the Python-tool setup both reach 100%, up from 48.6% for the base model. The CPU itself is reported as exact, so the misses that remain come from how Qwen uses it, not from the arithmetic.

Adding the computer did not measurably change how Qwen writes ordinary text: the reported KL divergence from the base model is 6e-4, against a noise floor of 5e-4.

The costs are listed plainly in the post. The checkpoint grows to 18.8 GB, roughly twice base Qwen, though mostly zero padding. It does not run with a plain generate() call. Every CPU instruction costs several decoded tokens, so a big calculation takes seconds; the batched benchmark executor managed about 3,200 CPU tokens a second on an NVIDIA L4. A Python sandbox does the same job faster. On Hacker News, some commenters asked what the point was, and one pointed back to Neural Turing Machines from 2014. The comparison is fair as far as it goes, with one difference: those models learned their memory access, while this one is compiled into an existing pretrained model.

There is also an experimental filesystem: file names hashed into 1,024 slots in the DeltaNet state, up to 512 files or about 53,000 tokens, and the state can be saved and loaded into a later conversation. On the author's test of facts that changed between chats, it scored 90%, against 30% for pasting the old chats back in.

The more interesting follow-up

The reason to care is in a companion post, which uses the same tooling to ask a safety question: can a hybrid model track state silently in its recurrent layers, without writing anything down where a chain-of-thought monitor could see it?

The test is parity, a light switch flipped many times. Models were fine-tuned to answer "on" or "off" with no reasoning, on up to 64 flips, then tested at 128. OLMo-Hybrid-7B, whose DeltaNet write strength can go up to 2, averaged 0.89 at 128 flips over six seeds. Qwen 3.5 4B, which caps beta at 1, fell to 0.55, about chance, as did the dense OLMo-3-7B at 0.54. Capping beta at 1 in the same OLMo-Hybrid model brought it down to 0.52 on all six seeds, for about a 3.5% perplexity cost on ordinary text. The author ties this to earlier theoretical work by Grazzi, Siems and colleagues, which argues that a linear RNN needs negative eigenvalues, and so a write strength above 1, to flip a bit on every token.

The author is careful about the limits: one task, six seeds for the main comparison and three for the rest, and answer-only fine-tuning standing in for real training pressure. Still, the implication is concrete. Whether a hybrid model can learn to carry hidden state across tokens appears to depend partly on an architecture parameter that Qwen and Ai2 set differently, and the compiled weights give interpretability researchers something rare: a mechanism planted at a known location, to check whether their tools can find it.

A CPU inside a language model is a strange way to do multiplication. As a way of showing what the recurrent half of a hybrid model can hold, exactly and on purpose, it is surprisingly clarifying.