TIS-100 Bench: Opus 5.5 holds 54 of the 133 new tradeoffs LLMs found in a decade-old puzzle game, and players beat its record in under three hours

Wincon Research gave 16 language models TIS-100, Zachtronics' 2015 assembly-programming puzzle game, and asked them to do what its players have done for a decade: make working programs smaller and faster. The write-up, "TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games" (November 3, on the Hacker News front page today), reports 133 new frontier points on 18 of 55 puzzles and three outright cycle records. Opus 5.5 and GPT-6 Astra share first place.

We re-ran the numbers from Wincon's published data and walked through the community leaderboard's commit history. The headline count holds up exactly. The ranking hides how lopsided the result is, and the human side of the story is faster than the write-up says.

What the benchmark measures

A TIS-100 puzzle is solved by a grid of tiny processors, each with two registers and room for 15 instructions, passing values to its neighbours. The game scores a solution on three numbers: cycles, nodes used and instructions. Lower is better on all three, and improving one usually costs another, so every puzzle has a Pareto frontier rather than a single best answer. Players have been pushing those frontiers for years on a community leaderboard kept on GitHub and updated by a bot.

Wincon froze that leaderboard on October 15 and gave each model the manual, the visible test cases and the game's feedback through text tools, but not the community's programs or exact records. Each model got one scored attempt per puzzle (pass@1), with 10, 30 or 45 minutes depending on difficulty, and a reminder to keep going if it tried to stop early. A "new frontier point" is a program that no community solution ties or beats on all three scores, replays in a fresh game and passes 100,000 random tests on the independent TIS-100-CXX simulator.

The other ten models, from Kimi K3.1 Pro down to MiMo-V2.6-Pro, found new points on two puzzles or fewer. Five found none.

Our recount: the 133 is right, the tie is not much of a tie

Wincon publishes every Pareto point its models found as a CSV, with the program files under MIT. It has 3,714 rows, 186 of them marked as new frontier points. Pooling them by puzzle and dropping any point beaten on all three scores by another model's point leaves exactly 133, the number in the headline.

The pooled frontier is also the most direct answer to "who found what", and it isn't close. Of the 133 surviving points, Opus 5.5 found 54. Gemini 4 Argon found 24, GPT-6 Astra 23, Sonnet 5.5 and Fable 5.1 11 each, GPT-6.1 Sol 8, Kimi K3.1 Pro 2 and Haiku 5.5 one. Only one point was found by two models independently.

The write-up's ranking counts puzzles with at least one new point, which is why Opus 5.5 and GPT-6 Astra tie at 11. It does mention that Opus found more points, 72 to 32, but those per-model counts include points another model beat; on the pooled frontier the gap is 54 to 23, and third-placed Argon contributed as much as Astra. Astra wins on record proximity, 78.0 to 77.2, and Wincon notes that the interval on that gap includes zero. Record proximity is also built so that each of the three scores can come from a different program, so it rewards being close on each axis, not finding new tradeoffs.

Both views are legitimate. They measure different things: Astra is consistently near the human records, while Opus more often finds a design the humans didn't have.

Where the new points are

All 186 marked points are on medium or hard puzzles, 167 of them on hard ones; none on the 19 easy puzzles, where community programs are likely close to optimal. All three outright cycle records came on puzzles from a 2018 community tournament rather than the main game. Wincon doesn't say why. A likely reason is that those eight puzzles have had much less attention than the 47 in the game itself, and if so, that matters when reading "LLMs beat a decade of human optimization": the biggest wins came where the humans had spent the least time.

The write-up is upfront that the puzzles are public and may overlap with training data. That cuts the other way here: a model that memorized community solutions would land near the existing frontier, not past it.

Three hours

The showcase is Undecimal Twin Rounder, a tournament puzzle. Wincon writes that the best community program at the freeze took 2,694 cycles, that Opus 5.5 finished at 1,918 (29% faster), and that player LongingForRest cut it to 1,744 "by the next day".

The leaderboard repository's history fills in the timeline. The bot logged Opus 5.5's 1,980-cycle program at 21:19 UTC on October 23 and its 1,918 at 21:29. LongingForRest's 1,848, credited to "LongingForRest/ opus-5-5", arrived at 23:45, and the 1,744 at 00:18 UTC on October 24: two hours and 49 minutes after the model's record. So yes, the next day, but only by UTC.

The same log shows that the live leaderboard had already moved before Opus's program landed. Earlier on October 23 the bot added 2,267 and 2,234 credited to "wincon" with no model named, then 2,112 credited to "wincon/LongingForRest". The 29% is measured against the frozen record, as the benchmark's rules say it should be; against the leaderboard as it stood that evening, Opus's 1,918 was about 9% faster. The write-up doesn't say who produced the unlabelled entries.

That pattern is everywhere in the log. Since October 15 the bot has added 329 entries to the TIS-100 leaderboard. By our count of the credit lines, 164 credit Wincon, 79 credit players building on a named model's program ("easone/opus-5-5", "Hersmunch/Sol-6.1", "12345ieee/gpt-6-astra") and 86 credit players alone. The leaderboard records the collaboration in its credit lines. Wincon is still submitting: the latest additions, six GPT-6 Astra programs for Prime Factor Calculator, were logged on November 11, eight days after the November 3 snapshot.

What to keep in mind

• One run per model and puzzle. The study can't tell you how much a model's score would vary on a second try.
• Settings differ. Grok 4.7 ran at low reasoning effort on 46 of 55 puzzles and medium on nine; the other models ran at high effort or their provider's default. A prompt nudge to test drafts early was used on 3 of 55 puzzles for GPT-6 Astra, 14 for Opus 5.5 and all 55 for Kimi K3.1 Pro, DeepSeek V4.1 Flash, Muse Wonder 1.0 and Muse Spark 1.3.
• Time and difficulty are tied. Hard puzzles got 45 minutes and easy ones 10, so "new points cluster on hard puzzles" can't be separated from "hard puzzles got more search time".
• Newer models are missing. Claude Fable 5.2, released November 9, and Gemini 4 Carbon aren't in the test. Fable 5.1, the most expensive model in it at an estimated $7.19 per puzzle, scored 70.2 on record proximity, a hair above Sonnet 5.5's 69.9 and well behind Opus 5.5.
• The tester is not neutral. Wincon describes itself as a lab that partners "with frontier labs to help them win via superior data and evaluations". Nothing in the published data looks off, but the benchmark is also a showcase.

Cost is the cleanest practical result. GPT-6.1 Sol reached 75.6 record proximity at about $0.75 per puzzle; Opus 5.5 reached 77.2 at $5.08. If you want a model near the best known answer, Sol is the bargain. If you want a design nobody has tried, the data says to pay for Opus.

We didn't replay the programs in TIS-100-CXX ourselves; our checks cover Wincon's published points and the leaderboard's own log. Wincon has since published a second benchmark in the same style, SpaceChem Bench, dated November 8.