Balestriero and LeCun turn the Platonic Representation Hypothesis into a theorem for JEPAs, and a 109M text model out-aligns 72B LLMs, with a captioner in the loop

Randall Balestriero (Brown University) and Yann LeCun (NYU, AMI Labs) posted Proving and Improving the Platonic Representation Hypothesis with JEPAs on October 29. The Platonic Representation Hypothesis (PRH), from Huh, Isola and colleagues in 2024, is the observation that vision and language models trained separately end up with increasingly similar representations as they scale. The new paper claims a proof of a version of it, and a cheaper way to get there than next-token prediction.

The claim

The "Platonic Representation Theorem" says that joint-embedding predictive architectures (JEPAs) trained separately, on any modalities, learn the same representation up to a single rotation, provided they agree on which observations are positives. The paper's term is "good views": two different observations of the same scene, for example two captions of one photo. "Bad views" are two corruptions of a single observation, such as two masked copies of one sentence. The authors argue that next-token prediction has the form of a bad-view JEPA, so a language model spends part of its embedding on its own modality's nuisance, which no rotation can line up with an image encoder.

The theorem has real fine print, and the paper states it: it holds for population optima, for "injective renderings of one world" and for objectives whose minimizers are unique up to rotation. One part is proved for spectral solutions only, and the authors note that no experiment meets the hypotheses exactly, since the runs use the LeJEPA objective and the image side mostly uses crops of one image as positives.

The headline numbers

The main result is a 109M-parameter BERT-B text JEPA trained on 900M samples from the texts of 940M web images in Recap-DataComp, about 116B text tokens. On the PRH paper's own WIT-1024 benchmark (mutual 10-nearest-neighbour agreement with a frozen DINOv2 ViT-B), it scores .184. The best language models the authors evaluated score .154 (Qwen2.5-72B) and .151 (Llama-3-70B), and CLIP's B/32 text encoder scores .124. Next-token prediction and a bad-view JEPA trained on exactly the same texts with the same encoder reach .111 and .088. The ordering holds against CLIP ViT-H/14 and on COCO.

The efficiency claim is that good views reach next-token prediction's 900M-sample score (.111) after 21.6M samples, about 42 times fewer (900 / 21.6 ≈ 41.7). Because a good-view sample reads three texts, that works out to 14 times fewer tokens.

Two demos show what the shared space is good for:

• Captioning. Frozen DINOv2 image embeddings pass through one linear map, fitted on 30k Flickr image–caption pairs, into a language model fine-tuned on text only. It scores CIDEr 0.838 on COCO.
• Pair-free classification. On the pair-free benchmark from Phillip Isola's group, the text JEPA gets 64% against at most 56% for the LLMs up to 8B reported there, on ImageNet-100.

The code is public under MIT, and there is a live demo.

Where the image information comes from

The phrase being shared most widely is that "no network was ever trained on image–caption pairs". That is true of the loss, but image information still gets in, and the paper says so itself. Its positives are texts grouped by the image they describe. On Recap-DataComp, two of each image's three texts were written by an image captioner, LLaVA built on LLaMA-3-8B, which sees images through CLIP ViT-L/14. The authors' own summary is that "paired information thus enters through the definition of positives, one fitted linear map and, on Recap, captioner-generated texts, never through a cross-modal loss."

The paper's Table 7 shows how much that matters. With no image captioner at all, pairing each alt-text with a paraphrase written by text-only Qwen2.5-7B, good views reach .060 on WIT, against .107 with the LLaVA recaptions on the same 2.24M images. On the two LLaVA recaptions alone at 50M images, good views reach .096 and bad views .092. Retrieval on Flickr shows a clearer gap (R@1 .386 against .325). So the comparisons with Qwen and Llama at scale depend on captioner-written text, and the authors acknowledge this. The view-versus-view comparisons, run on identical texts, do not depend on it.

Nicolas Dufour raised the same point under the announcement:

Balestriero replied that next-token and bad-view baselines on the same corpus show that the views drive the alignment, "not the training corpus alone". He also agreed that "we need a better way to generate positive views for text".

The paper flags a few more caveats. The .184 headline comes from the first seed. A fourth seed, trained with twice the effective regularizer weight, reached only .121, although all four seeds average .168, still above every language model tested. The bad-view control at this scale has one seed. And at least 43 of the 1,024 WIT captions also appear among Recap-DataComp alt-texts; the authors report that removing them moves none of the text models they checked by more than .002.

What to watch

The theory is the durable part, and it gives a testable recipe: decide what counts as a positive and train for it. The open question is the text side without a vision-language captioner. The headline comparison with 72B LLMs has so far been shown only on recaptioned image descriptions, not on ordinary web text such as Common Crawl. That is the experiment that would show whether "good views beat scale" holds beyond descriptions of images.