NVIDIA's looped Dory1.0-4B: more loops lift coding and agent scores, not knowledge, and the win over Gemma 4 26B-A4B rests on one hallucination metric

NVIDIA's Nemotron Labs has published the technical report for Dory1.0-4B, the looped 4-billion-parameter model it put on Hugging Face on October 16. "Dory1.0-4B: A Looped Normalized-GPT" (arXiv 2611.03998, posted November 2) is by Maxim Fishman and Simeng Sun, who share first authorship, with Shantanu Acharya, Faisal Ladhak, Brian Chmiel, Santiago Akle Serano, Ron Banner, Boris Ginsburg and Krishna C. Puvvada. Release-day posts mostly repeated the model card's pitch: a 4B model that beats bigger ones if you let it think in loops. The report has the full table behind that pitch, and the table is more interesting than the slogan. Looping clearly helps on agentic and coding tasks, it does nothing for what the model knows, and the "outperforms larger models" average leans heavily on one metric.

What a looped model is

Dory stacks 72 sublayers (attention and MLP counted separately) into three groups: a 16-sublayer input block, a 40-sublayer recurrent block and a 16-sublayer output block. At inference the recurrent block can run once, twice or four times per token, with the input re-injected on every pass. The parameter count stays at 4B; only the compute per token changes. Because NVIDIA trained the model with a mixture of loop counts, one checkpoint serves every setting, and you choose the loop count when you launch the server:

vllm serve --model nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 \
    --hf-overrides '{"n_recurrent_loops": 2, "recurrent_kv_cache_mode": "per_loop"}'

(Trimmed from the model card's launch command.) The backbone is a normalized Transformer (nGPT) with a 3:1 mix of sliding-window and full-attention layers and a 256K context. Of the 6 trillion pretraining tokens, the first 4.5 trillion were trained natively in NVFP4, NVIDIA's 4-bit floating-point format. Post-training used SFT and then RL with a "difficulty-aware loop curriculum" that sends harder prompts to deeper loops. The weights come in BF16, MXFP8, NVFP4 and GGUF under the OpenMDW-1.1 license.

Where the loops pay off

The agentic numbers move a lot. All of them are NVIDIA's own runs in its NeMo Evaluator harness, so they are not independent results:

Two loops buy most of the gain, and four add a little more on agentic tasks. Reasoning-heavy exams move less: on GPQA Diamond and Humanity's Last Exam, Dory stays behind Qwen3.5-9B at every loop count. Knowledge doesn't move at all. AA-Omniscience accuracy sits at 10.0% whether the block runs once or four times, half of what the larger models score. That matches what a looped model is: extra passes give the same weights more computation per token, but the facts still have to fit in 4B parameters.

What the average rests on

The headline claim is that with two or more loops, Dory beats Qwen3.5-9B, Gemma 4 12B and Gemma 4 26B-A4B "on average across 12 benchmarks". We recomputed the equal-weight averages from Table 2 and they match the paper: 39.67 at two loops and 40.60 at four, against 32.21 for Qwen3.5-9B, 32.59 for Gemma 4 12B and 36.73 for Gemma 4 26B-A4B.

One of the twelve columns, though, is AA-Omniscience non-hallucination. The paper defines it as one minus the share of wrong answers among all answers that weren't correct, abstentions included. A model that says "I don't know" instead of guessing scores high, and Dory does: 79.74 at two loops, against 17.39 for Qwen3.5-9B and 16.36 for Gemma 4 26B-A4B. That is a real and useful property in a small model that knows few facts. But a 60-point gap in a single column moves an equal-weight average by about 5 points.

Take that column out and recompute the other eleven, and Dory at two loops scores 36.0. That is still ahead of Qwen3.5-9B (33.6) and Gemma 4 12B (33.6), but behind Gemma 4 26B-A4B (38.6). At four loops it scores 37.1, still behind. So the win over the two dense models holds without it, and the win over Gemma's 26B mixture-of-experts model depends on it. The average also turns GDPval Elo into a percentage-like score as (Elo − 500)/20, which gives Dory's 946 a 22.3 against 2.3 for Qwen3.5-9B. The paper discloses both choices; the release-day summaries didn't mention them.

Fewer tokens, more compute per token

NVIDIA also says looping makes the model more token-efficient: on four agentic benchmarks, four loops generate 1.3 to 2.5 times fewer tokens per attempt than one loop. That's true, and the report itself calls it "partially offsetting the higher compute per token". Our arithmetic from the architecture: one loop runs 72 sublayers per token and four loops run 16 + 4×40 + 16 = 192, about 2.7 times as much (ignoring embeddings and context-length effects). Even at the best case of 2.5 times fewer tokens, four loops end up roughly even on compute per task, and at 1.3 times they cost about twice as much. The loops convert compute into accuracy; they don't make it free.

Memory is the other cost. By default each loop keeps its own KV cache, so cache memory grows with the loop count. Dory also supports last-loop caching, in which each loop overwrites the previous one's entries, so the cache stays at one-loop size. The abstract says this keeps "most of the accuracy gains". Table 6 puts the cost at four loops at between −2.1 and +0.4 points per benchmark, with the largest drop on AA-LCR, the long-context test (72.94 to 70.81). The model card's own vLLM example still uses per_loop. NVIDIA also uses the loop dial for self-speculative decoding: the same checkpoint drafts at one loop and verifies at four, which it measures at 1.37 to 1.71 times the decode throughput of plain four-loop decoding.

Running it

vLLM support (PR #60149) was merged on October 25 and is listed in the v0.33.0rc3 release candidate of November 2. The model card still tells users to install vLLM from the PR. The llama.cpp port (PR #30306, by NVIDIA's Brian Chmiel) is still an open draft. Its description warns that last-loop caching "changes predictions compared with separate caches, and prefill chunk size matters", and lists Metal as untested and loop counts above 4 as unsupported. That is worth knowing before you compare a GGUF run with NVIDIA's numbers.

Looped and recurrent-depth models are not new. The report cites Universal Transformers and the more recent recurrent-depth work as its lineage. What Dory adds is a fully post-trained, openly licensed checkpoint where the depth dial can be tested on real agentic tasks. The next useful step is someone outside NVIDIA running it on SWE-Bench Verified and Terminal-Bench at one, two and four loops and measuring wall-clock cost, not just tokens.