Anthropic's FlashProve: zero-knowledge proofs for Llama 4 Maverick at 110 tokens/s, and a strict check stock engines fail

Anthropic has released FlashProve, an open-source system that lets a model provider attach a cryptographic proof to a reply showing that it came from one specific, committed set of weights, without revealing those weights. The research post, "Progress on provable inference for large language models", comes from Anthropic's Frontier Red Team: Roy Rinberg and Alex Allain of Anthropic, with Alessandro Chiesa (EPFL, StarkWare), Miranda Christ (MIT), Rahul Dalal, Giacomo Fenzi (EPFL), Ben Mayer and Guy Weissenberg (EPFL). The post is dated November 2; Anthropic announced it on X on November 3. Code, a Lean formalization and documentation are on GitHub under Apache 2.0.

The headline number is speed. On Llama 4 Maverick, Meta's 400B-parameter mixture-of-experts model, in a public 4-bit conversion, one H100 generates proofs at about 110 tokens per second (a batch of 24 requests of 1,024 tokens in about 223 seconds). Each proof is about 18.78 MB and takes about 17 seconds to check on a 24-thread CPU, with no GPU and no access to the weights. Anthropic puts the cost at roughly 70 to 95 times the GPU time vLLM needs for the same requests.

That is a big jump for "zkML", the field of zero-knowledge proofs for machine learning. The post's own comparison: VerInf, the only earlier proof of Maverick the authors know of, reported 14.3 hours for 1,000 tokens and a 93.6 GB proof that took almost a day to verify. zkLLM in 2024 managed a 13B model at about 2.5 tokens per second in a batch. The closest number is commercial: Attestable reports 53 to 77 tokens per second for the dense Gemma 4 31B, without an open-source release.

What is being proved

FlashProve is a zkSNARK (a zero-knowledge succinct argument) built the standard way: an interactive oracle proof compiled with hash functions. The provider first publishes a commitment, a short fingerprint of the weights; that takes about 39 minutes once per model. After that, for a prompt and a reply, the provider can prove that the committed weights, run on that prompt, produce those tokens. The architecture is not hidden, only the weights.

Two engineering choices make it fit on one GPU. The prover's work scales with the roughly 17B parameters that a token actually activates, not all 400B, while the proof still hides which experts ran. And proving happens after the fact, in batches, in one parallel pass like prefill, with the weights streamed from about 290 GB of host memory.

The uses Anthropic lists are the interesting part: an independent evaluator checking that the model it tested is the model being served, a customer of a routing service checking that it got the model it paid for, a company detecting a compromised inference server, and a regulator checking that a provider's hardware serves an approved model. We wrote last week about OpenAI's Private Inference, which relies on trusted hardware for attestation. FlashProve is the software-only alternative: its guarantee rests on cryptographic hardness, not on trusting a chip vendor. Anthropic says the two approaches are complementary.

The post also says that Claude did much of the work: it "produced improvements on zkSNARKs for inference" and wrote the protocol specification. The repository's limits page goes further: "AI systems, under human direction, wrote nearly all of the software, the Lean proofs and the documentation, and ran the tests reported here." To make that auditable, the team states the security properties in Lean and ships an executable Lean verifier next to the fast C++ one.

The fine print

The research post lists its limits: proofs cover an integer version of the model under greedy decoding, contexts of up to 1,024 tokens, and no side-channel protection. The repository's Known limits page is far more candid, and it is the part worth reading.

Real inference engines don't pass the strict check. FlashProve has two acceptance modes. "Exact match" means the reply is exactly the integer model's greedy output, and that is the mode the soundness theorem covers. But "of the 96 honest replies scored from each of two stock engines none would pass exact match." In one measurement, the integer model's top token differed from the floating-point model's at 6.81% of positions. So serving relies on a looser "clipped-gap" policy, and of that the page says an accept "by itself bars little: with loose limits, close scores or flat (all-zero) weights almost any output passes." In one test, 28 of 32 replies with one token swapped for the model's second choice still met limits fitted to honest replies. The project's own advice: "if you cannot assess them, accept only exact match."

A proof names a commitment, not a model. "A yes does not say whose weights they are." A provider can commit to any model, including a small or degenerate one, so the check is only as good as the source of the commitment ID, which must come from a party the user trusts, never from the server being checked.

The proved parts are narrower than the programs. Soundness is machine-checked for the slow Lean verifier, at about 2^-124 per hash query with a floor of about 2^-101. The C++ verifier people would actually run and the CUDA prover are tested against the protocol, not proved. The page says weight hiding is expected rather than known, that "few attacks were sought", that there has been no independent security audit, and that two published attacks on Fiat–Shamir transformations (ePrint 2025/118 and 2026/1838) have not been examined against the protocol. It also notes that proofs are malleable and that a commitment runs out: 688,127 batches at full size, roughly the "16 million requests" in the blog post.

None of this is hidden; Anthropic calls the release "intermediate progress" and a research prototype, and the repository says it is not maintained. But it does change how to read the result. The 110 tokens per second is real and a large step for the field. What it buys today is a strong guarantee for an exact, greedy, integer, short-context computation, plus a weaker and still loosely specified one for what a production server actually emits.

Where this fits for Anthropic

A footnote ties FlashProve to Anthropic's Frontier Safety Roadmap, which set a goal of a provable-inference prototype by September 30, 2026. Anthropic says it met that goal with a different project, one that signs its inference traffic and recomputes inference on a trusted server, and on October 19 it renamed the goal "inference verification". FlashProve is a separate, cryptographic route to the same aim. It runs on an open Meta model; nothing in the release says it has been applied to Claude.