Will DePue's Blackwell fingerprint, rebuilt: DeepSeek-V4-Flash and Kimi K3 weights fit only 148- and 152-SM GPUs

Will DePue's claim that DeepSeek, Moonshot AI, MiniMax and Ant Group built their newest models on export-controlled NVIDIA Blackwell GPUs rests on a mechanism simple enough to check on an ordinary CPU. So we checked it. We wrote our own model of how PyTorch fills a tensor with random numbers on an NVIDIA GPU, pulled the untouched embedding rows of deepseek-ai/DeepSeek-V3-Base, deepseek-ai/DeepSeek-V4-Flash and moonshotai/Kimi-K3 straight from Hugging Face, and asked which GPU layout reproduces them. The answer matched his report every time: 132 streaming multiprocessors (Hopper) for DeepSeek-V3, 148 for DeepSeek-V4-Flash and 152 for Kimi K3, with nothing else even close.

That settles the forensic trick, not the politics. And it says nothing either way about the parts of the story that rest on weaker evidence or on arithmetic.

What the report says

The report, "Top Chinese Frontier AI Models Carry the Fingerprint of Export-Controlled NVIDIA Blackwell GPUs," was published by Unemployment Labs, DePue's small nonprofit research group, on November 12. DePue previously worked at OpenAI on Sora and on post-training for o3 and GPT-4o. He names six open-weight models whose starting weights match Blackwell: DeepSeek-V4-Flash and MiniMax-M3 match the B200/B300 layout (148 SMs); DeepSeek-V4-Pro, Kimi K3, Kimi K3.1-Pro and Ant Group's Ling-3.1-flash match GB200/GB300 (152 SMs). Earlier models from the same labs, including DeepSeek-V3, Kimi K2.5, MiniMax-M2.7 and Ling-3.0-flash, read as Hopper. The newest readable models from Alibaba, Zhipu, Tencent, Xiaomi and StepFun all read as Hopper too.

How a GPU leaves its fingerprint

Before training, PyTorch fills each weight tensor using cuRAND's Philox generator. The generator is deterministic: the same seed gives the same numbers on any chip. What changes is where those numbers land. In PyTorch's CUDA kernel (aten/src/ATen/native/cuda/DistributionTemplates.h), the grid size for a large tensor is capped at the device's multiProcessorCount times maxThreadsPerMultiProcessor / block_size. On the chips in question that comes to 2,048 threads per SM. Each thread draws its own Philox stream and writes every G-th element, where G is the total thread count: 270,336 on a 132-SM Hopper, 303,104 on a 148-SM B200/B300, 311,296 on a 152-SM GB200/GB300. Same seed, different SM count, and the same numbers are shuffled into different positions.

That would be useless if training scrubbed every starting value. It doesn't. Rows of the embedding matrix that belong to tokens the model never sees in training never get a gradient, so they ship with their initial values intact.

Our check

We didn't run DePue's code. We wrote a CPU version of Philox4x32-10 (validated against the Random123 test vector) and of cuRAND's Box-Muller transform, followed the thread layout in PyTorch's kernel, and fetched the rows ourselves with HTTP range requests against the pinned revisions his code lists. The only things we took from his repository, macrogrants/checkpoint-forensics, were the seeds, the Kimi generator offset and which rows to read: 953049583 for both DeepSeek models, and 20256586 at offset 3776 for Kimi K3.

The figures are correlations between the published row and the regenerated one. A full sweep of every SM count from 16 to 256 on one row per model gave a single peak at 1.0 (132, 148 and 152 respectively), and the runner-up never got past 0.043. Kimi K3 also fails with the right seed and layout but the wrong generator offset, which is what you would expect if the values really came out of that one training job's random stream.

The match goes down to individual values. We scaled our regenerated numbers by the 0.006 standard deviation DeepSeek used and rounded them to bfloat16. Across the 14 DeepSeek rows, 30 of 81,920 values differed from the published weights, and three rows came out bit-identical. Twenty-seven of the 30 were one bfloat16 rounding step off. The other three were tiny values near zero, where small float32 differences between numpy's sin and log and the GPU's versions show up most. DePue reports 445 DeepSeek-V4-Flash rows matching bit for bit when he ran stock PyTorch on an actual B300.

Two rows we picked as controls, 1000 and 50000, belong to ordinary tokens that the model did train on. At the right layout they still correlate weakly with their starting values: 0.19 and 0.08 in DeepSeek-V3, 0.14 and 0.003 in V4-Flash. At the wrong layouts they correlate with nothing. That faint survival is the "partial survival" evidence the report uses where no untouched rows exist. We also hit one row that training had partly overwritten: V4-Flash row 129279 still correlates at 0.999, but only 4.5% of its values are identical.

What the check doesn't settle

The headline model has the weakest evidence. DeepSeek-V4-Pro, the first model named in DePue's thread, is one of the four GB200/GB300 readings. Its report entry says "Partial survival, 49σ": it has no untouched rows. In a reply on X, DePue wrote that "only one of the six rests on trained weights alone, that's v4 pro." His repository includes rows for the other five, not for V4-Pro. Forty-nine sigma is still far above the roughly 8 sigma that the report expects chance to produce across six trillion seed and layout combinations. But it rests on his statistical model, not on a replay anyone can run in a few seconds.

A layout is not a chip. The report itself shows an AMD MI300X limited to 148 or 152 of its compute units reproduces the same layouts. It rules that out only for DeepSeek-V4-Flash: of 377 values where AMD's and NVIDIA's arithmetic differ, 363 match NVIDIA and none match AMD. For the other five models it calls AMD "very unlikely" rather than excluded. A CPU check can't help here.

Initialization is not training. Security researcher Daniel Franke argued on X that the claims depend on "assuming each model was trained on the hardware which initialized it," since generating starting weights is cheap. DePue answered that 24 of 24 models with publicly disclosed hardware read correctly, which would be unlikely if labs routinely initialized on one machine and trained on another. He added that Kimi K3's weights were created by "at least 64 separate processes," all with the 152-SM layout. That is a strong argument, but it is an argument rather than a measurement, and the report says as much: the fingerprint "supports, without proving" Blackwell training.

The thread is stronger than the report. The thread calls the evidence "conclusive" for all six models. The report reserves "conclusive" for DeepSeek-V4-Flash and calls the rest "strong evidence." The cluster estimates also moved on publication day. The thread said 2,000 to 3,000 Blackwell GPUs for the largest runs and $150 to $200 million per cluster. The revised report says about 4,500 for Kimi K3 and 3,000 for DeepSeek-V4-Pro, with ranges as wide as 1,000 to 9,500, and adds, "We are not confident in these estimates." Those numbers rest on guessed run lengths and utilization, not on the weights.

Location, ownership and legality are out of scope. Blackwell systems can be rented outside China, and the report says its results "are not a finding that any law was broken."

The fingerprint is easy to wipe

DeepSeek-V4.1-Flash, released in September, shows no signature. DePue says its untouched rows are still there, but none of about 4.3 billion seeds matches them under any GPU layout from 16 to 512 SMs, and neither does PyTorch's CPU generator. His conclusion is that DeepSeek changed how it creates starting weights between April and September, and that this says nothing about the chip.

What it does show is how fragile the method is. The report notes that several Qwen3 models were initialized on the CPU and carry no GPU layout at all. In our reading, any lab that wants to stop leaking this signal only has to initialize on the CPU, change its seeding scheme, or overwrite never-trained embedding rows before release. None of those costs anything. Checkpoint forensics is a good way to read the hardware history of weights already on Hugging Face. As a monitoring tool for future releases, it lasts only until labs notice.

A small oddity in the appendix: MiniMax-M3's embedding uses seed 2760 (plus the shard number), and 2760 is also the seed the report recovered for Baidu's ERNIE-5.5, which was built in a different framework. The report doesn't comment on it, and a short seed shared by two labs may be nothing more than a coincidence.