Prefrontal's Latent Sparse Attention speeds up one attention layer 72x at 10M tokens; end to end, the gain is 9x

Sparse attention was supposed to fix the cost of long context: each query looks at only a handful of earlier tokens instead of all of them. The catch is that something still has to pick those tokens, and in today's designs that picker scores every preceding key. According to Prefrontal AI, a London research lab, at 10 million tokens that search eats 98% of a sparse attention layer's time.

Prefrontal's answer is Latent Sparse Attention (LSA), released on November 2 with a paper, Apache-2.0 code, GPU kernels for H200 and B300, and an SGLang fork. The paper, "Latent Sparse Attention: Towards Subquadratic Ultra-Long Context Attention" (arXiv 2610.42147), is by Ales Kubicek, Andrea Caciolai and Liangzu Peng (core contributors), Sukrit Sumant, Benjamin Hoover, Fraser Greenlee, Ruairidh Battleday, Radim Urban and Torsten Hoefler of ETH Zurich, with James Whittington as senior author. Whittington announced it on X:

What it changes

The baseline LSA targets is the indexer, the small scoring network that DeepSeek introduced with DeepSeek Sparse Attention (DSA) in V3.2 and that GLM-5.3-Flash also uses. An indexer is cheaper than attention (fewer heads, smaller dimension, FP8), but it still compares every query with every earlier key, so its cost stays quadratic.

LSA groups keys into clusters in a learned latent space, and groups those clusters into a hierarchy. A query scores a few clusters at each level, walks down the tree and only then fetches individual keys. Attention itself runs over a fixed number of keys whatever the context length. With the full hierarchy, the authors say, prefill goes from O(T²) to O(T log T). The attention kernel is untouched: LSA only decides which keys get read.

It comes in two forms. D-LSA is bolted onto an existing model by distillation: only the new indexer projections are trained, on 352M tokens, while the model stays frozen. N-LSA is trained from scratch with ordinary next-token loss.

The numbers, and which ones to quote

The 72.4x in the announcement is real but narrow. It is one attention layer of Qwen-3.8-27B, prefilling the last 32k chunk of a 10M-token context on one H200, against FlashAttention-3 full attention. Run the whole model in SGLang on eight H200s and the time to first token improves 9.2x; time per output token improves 2.9x.

The gains also depend heavily on length. Here is Qwen-3.8-27B on H200, from Prefrontal's tables:

At 1M tokens on B300, per-token generation is actually slower than the baseline (0.7x), which Prefrontal footnotes as missing optimized baseline kernels on Blackwell under parallelism. The paper says that below 1M the single-layer prefill gain drops to 2.1x at 512k. In other words, this is a technique for contexts most people do not run yet.

Against a model that already has sparse attention, the margin is smaller. Replacing GLM-5.3-Flash's DSA indexer gives 22.7x per layer in prefill and 4.6x in decode at 10M tokens, but 2.2x time to first token and 2.1x per output token end to end.

What it costs in accuracy

On short benchmarks, the distilled models stay close: Qwen-3.8-27B moves from 83.2 to 83.0 on MMLU-Pro and 98.3 to 97.5 on AIME25. Long context is where the differences show. Qwen keeps pace with the original on RULER up to 10M tokens, but drops from 70.0 to 65.0 on LongBench v2, which the authors say is inside the 95% confidence interval. GLM-5.3-Flash with LSA trails its baseline on RULER by up to about 8 points between 2M and 5M tokens, and by about 6 to 7 points on Prefrontal's own new benchmark, PaperTrail, from 2M to 10M. The GLM baseline was distilled and fine-tuned; the LSA version was only distilled, so the comparison is not entirely like for like in either direction.

PaperTrail itself is worth a look. It hides chains of synthetic facts in a pile of recent papers (a method renamed later, citing papers quoting the old value) and asks questions that require following the chain, including ones whose correct answer is "not in the papers". That is closer to literature search than the needle-in-a-haystack tests that dominate long-context evaluation, though it is new and comes from the same team.

Native training, at small scale

The N-LSA result is the more ambitious claim and the least tested. In a roughly 0.5B-parameter hybrid model trained at up to 128k context, N-LSA reached the same validation loss as full attention (1.085), ahead of sliding-window attention (1.153), NSA (1.117) and PISA (1.100), and trained faster: the abstract says 1.6x per layer at 128k, and the end-to-end training chart shows 1.24x. Whether that holds at the scale of the models it is distilled into is an open question the paper does not answer.

The release is unusually complete for a lab's first open research: code, kernels and the serving integration are all public. The headline number, though, is the best cell of a table. For anyone serving models at today's 128k to 1M contexts, the relevant figure is closer to 1.5x on time to first token, and on decode, nothing yet.