SenseNova-Loop-9B-Preview is Qwen3.5-9B with 24 layers run twice: we compared the weights
SenseTime's SenseNova-Loop-9B-Preview, the "looped" open model it released on Hugging Face as sensenova/SenseNova-Loop-9B-Preview on October 30, is built on Alibaba's Qwen3.5-9B. Not just the architecture: the weights. We compared the two checkpoints tensor by tensor, and every one of the 32 decoder layers lines up with the Qwen3.5-9B layer at the same position, while the vision encoder is byte-for-byte unchanged. The model card, the GitHub README and the suggested citation never say the weights come from Qwen (Qwen models appear only as competitors in the benchmark chart), and the card's metadata has no base_model field.
That doesn't make the model less interesting. It does change how to read its benchmark chart, which puts SenseNova-Loop next to Qwen3.5 9B as if they were two independent 9B models.
What SenseTime describes
The card describes a looped decoder: layers 0–3 run once, the block of layers 4–27 runs twice with the same weights, then layers 28–31 and the output head finish the job. That is 56 layer executions per token from 32 stored layers. Inside the loop, two residual streams are mixed by mHC (manifold-constrained hyper-connections, the scheme DeepSeek published in arXiv:2512.24880), with a Sinkhorn-normalized mixing matrix. The card calls the result "~9B total parameters; ~14.6B inference parameters" and pitches it at long-context, agentic and office work. SenseTime announced it as a research-track side release next to its SenseNova 6.8 Flash product model.
𝗦𝗲𝗻𝘀𝗲𝗡𝗼𝘃𝗮 𝟲.𝟴 𝗙𝗹𝗮𝘀𝗵 𝗶𝘀 𝗵𝗲𝗿𝗲!
𝗥𝗲𝗹𝗶𝗮𝗯𝗹𝗲 𝗪𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀. 𝗕𝗲𝗮𝘂𝘁𝗶𝗳𝘂𝗹 𝗗𝗲𝘀𝗶𝗴𝗻.
🧠 𝗢𝗳𝗳𝗶𝗰𝗲 𝗜𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲: 𝗟𝗼𝗻𝗴-𝘁𝗲𝗿𝗺 𝗺𝗲𝗺𝗼𝗿𝘆 & 𝗽𝗿𝗼𝗮𝗰𝘁𝗶𝘃𝗲 𝗶𝗻𝘁𝗲𝗿𝗮𝗰𝘁𝗶𝗼𝗻 𝘁hat remember your needs and keep tasks… pic.twitter.com/thOK7BDPqm
The code is not shy about where the architecture comes from. config.json declares the text model as "model_type": "qwen3_5_text", the remote code subclasses Transformers' Qwen3_5Config and Qwen3_5TextModel, and the open serving pull requests say the same: vLLM #64832 adds support "reusing Qwen3.5 layers", and SGLang #47281 describes the model as "based on Qwen3.5". The shapes match Qwen3.5-9B exactly: hidden size 4,096, feed-forward 12,288, 32 layers in a three-linear-attention, one-full-attention pattern, and a 248,320-token vocabulary.
Same architecture is not the same weights, though. A lab can reuse a public architecture and train from scratch. So we checked.
What we checked
Using HTTP range requests against the safetensors files, we read the first million values of matching tensors from the current SenseNova-Loop checkpoint (revision 89d7970, after the November 14 commit that converted the backbone to BF16), from Qwen/Qwen3.5-9B and from Qwen/Qwen3.5-9B-Base, and computed cosine similarity.
- Tensor list. SenseNova-Loop has every one of Qwen3.5-9B's 775 tensor names, plus 96 new ones:
mhc_attnandmhc_ffnweights and biases for each of the 24 looped layers. Total parameters are 9,656,250,480 against Qwen3.5-9B's 9,653,104,368. The difference, 3,146,112, is exactly the mHC routers. Nothing else was added or resized. - Decoder layers. For
mlp.gate_projin all 32 layers, cosine similarity with the Qwen3.5-9B layer at the same index runs from 0.940 to 0.969. Spot checks on attention projections, down/up projections, norms, the embedding table andlm_headgive 0.955 to 1.000. As a control, the same tensor compared with a different Qwen layer (SenseNova layer 10 vs Qwen layer 14, layer 20 vs 21) gives 0.001 and −0.0004. Independently trained weights would look like the control. - Vision encoder. Sampled tensors from the first and last vision blocks and from the merger that projects image features into the language model are bit-identical to Qwen3.5-9B. The vision side appears to have been frozen.
- Which Qwen. In all 32 layers, SenseNova-Loop is slightly closer to the released Qwen3.5-9B than to Qwen3.5-9B-Base (by 0.001 to 0.004). That consistent margin suggests the starting point was the post-trained model, though it is small enough that we would not call it proven.
The training was not light. Qwen's own post-training moved the same tensors very little (Base vs released model: 0.997 to 0.999), while SenseTime's retraining moved them far more (0.94 to 0.97), most of all in the middle of the loop, layers 12 to 19. Continued training with the loop and mHC switched on reshaped the network; it just started from Qwen.
Our arithmetic also explains the "~14.6B inference parameters" figure: counting the 24 looped layers twice gives 14.85 billion, and dropping the 243-million-parameter MTP head (used for speculative decoding) leaves 14.61 billion.
Why the chart reads differently now
SenseTime's benchmark chart compares SenseNova-Loop-9B-Preview with Qwen3.5 9B, Qwen3.5 27B, Qwen3.5 and Qwen3.6 35B-A3B, Nanbeige 4.2-3B and IQuest-Coder-V1-40B-Loop-Thinking. Social posts summarized it as a 9B model that matches 27B models. Knowing the lineage, the Qwen3.5 9B column becomes a before-and-after:
| Benchmark | Qwen3.5 9B | SenseNova-Loop-9B-Preview | Qwen3.5 27B |
|---|---|---|---|
| BIRD-Dev | 68.48 | 71.38 | 69.84 |
| WorkBuddy Bench | 41.63 | 66.89 | 60.53 |
| AIDABench | 40.78 | 58.87 | 57.19 |
| TeamBench | 33.33 | 48.89 | 44.44 |
| Terminal Bench 2.1 | 29.20 | 40.45 | 47.19 |
| τ³-Banking | 9.47 | 18.56 | 19.59 |
| AutomationBench | 1.05 | 13.72 | 10.17 |
| RaccoonPPTBench | 33.60 | 70.70 | 59.08 |
Values as read from SenseTime's chart on the model card. A footnote describes RaccoonPPTBench as an internally developed presentation-generation benchmark.
The gains over the starting checkpoint are large on office and agent tasks, which is exactly what the card says the training data was: long sequences, multimodal inputs and "supervised task trajectories". What the chart cannot tell you is how much comes from the loop. There is no control in which the same Qwen3.5-9B gets the same training without looping or mHC, and the technical report is listed as "TBD". On the two benchmarks closest to general agentic coding and tool use, Terminal Bench 2.1 and τ³-Banking, Qwen3.5 27B still finishes ahead, and on Terminal Bench so does the 3B Nanbeige model (49.44).
This is the same question we raised about NVIDIA's looped Dory1.0-4B, where more loops helped coding and agent scores but not knowledge. Dory at least shipped a report with loop-count ablations. SenseNova-Loop ships one fixed setting: the checkpoint is configured for exactly two passes (counts: [2], eval_count: 2), and its remote code supports only fixed-count checkpoints.
What the loop costs at inference
Reusing weights saves memory for weights, not for computation. Each token passes through 56 layers instead of 32, so decoder work rises by 75%. The KV cache grows too. Both the Hugging Face code and the two serving PRs keep a separate cache slot for every visit to a layer ("Repeated visits share weights, but each visit has its own cache slot"). Qwen3.5-9B has 8 full-attention layers. SenseNova-Loop executes 14: layers 3 and 31 once, and six full-attention layers inside the loop twice each. With 4 KV heads of dimension 256 in BF16, that is 4 KiB per token per full-attention execution. A full 262,144-token context needs about 8 GiB of KV cache on Qwen3.5-9B and about 14 GiB on SenseNova-Loop, per sequence. The fixed-size linear-attention state grows by the same 42-to-24 ratio, but that is tens of megabytes.
Serving support is also still unofficial. The model card's vLLM and SGLang instructions check out unmerged pull requests. Two weeks after release, the vLLM PR is labeled needs-rebase and the SGLang PR's own description says task-level accuracy and speed benchmarks are pending.
The upshot
SenseNova-Loop-9B-Preview is a real experiment: someone took a strong small model, ran most of its depth twice, added DeepSeek-style stream mixing, and trained it hard on agent work. The weights are Apache 2.0, as are Qwen3.5-9B's. But anyone weighing "a 9B looped model that rivals 27B" should know the 9B came pre-trained by Qwen, and the comparison that would isolate the loop has not been published.
- SenseNova-Loop-9B-Preview model card
- SenseNova-Loop-9B-Preview config.json
- modeling_sensenova_loop.py
- Qwen/Qwen3.5-9B
- Qwen/Qwen3.5-9B-Base
- vLLM PR #64832: Add SenseNova Loop support
- SGLang PR #47281: sensenova_loop multimodal Loop and mHC inference
- SenseTime announcement on X
- OpenSenseNova/SenseNova-Loop on GitHub
- mHC: Manifold-Constrained Hyper-Connections (arXiv:2512.24880)
Comments
No comments yet. Be the first.