OpenAI's BioBasicsBench: GPT-6 Astra reads raw wet-lab data at 66.7%, but GPT-6.1 Sol is statistically level at a fifth of the output cost

OpenAI has a new biology benchmark, and it is about the unglamorous part of lab work: reading a gel, counting colonies, noticing that a control plate is the wrong control. BioBasicsBench, published November 6 and built with the enzyme-design company Ligo, gives AI agents 100 routine wet-lab tasks drawn from Ligo scientists' own experiments. At OpenAI's xhigh reasoning setting, GPT-6 Astra earns 66.7% of the available credit and GPT-5.6 Sol 43.8%.

That 23-point jump is the headline. The table underneath says a little more, and so does what is missing from it.

What the benchmark asks

The inputs are the evidence a bench scientist actually looks at, not the cleaned-up figures that end up in papers: agarose and SDS-PAGE gels and Western blots (31 tasks), protocols and photographed lab notes (27), CSV and flow-cytometry exports (20), photos of benches and reagent tubes (18), raw microscopy including z-stacks and time-lapse (14), long-read FASTQ files (12), and even seven videos of bench procedures such as streaking a plate or setting up an SDS-PAGE tank. Tasks span protein methods, imaging and cytometry, assays, nucleic-acid methods and molecular cloning.

Answers are open, not multiple choice, and graded against expert-written rubrics, so a correct conclusion with a wrong measurement gets partial credit. Each task was reviewed by a Ligo scientist who had not written it, then by an external PhD-level wet-lab scientist, then once more internally.

That design choice matters more than it looks. The best-known earlier benchmark in this space, FutureHouse's LAB-Bench (Jon M. Laurent, Joseph D. Janizek, Andrew D. White, Samuel G. Rodriques and colleagues, 2024), was multiple choice, and its authors found that models scored above chance on the hard Cloning Scenarios largely by eliminating weak distractors and guessing. In one example, models picked ampicillin as a resistance marker "simply due it being the most common antibiotic used in bacterial selection experiments." When the same cloning questions were asked open-ended, Claude 3.5 Sonnet and GPT-4o each got 2 of 10 right. Rubric-graded open answers are OpenAI's way around that problem, at the cost of depending on how good the rubrics are.

There is a pleasing echo, too: one of BioBasicsBench's showcase tasks turns on exactly that antibiotic. GPT-5.6 Sol read an empty negative-control plate as a clean control. Astra noticed that the plate's rim label said "LB Kan50", kanamycin, while the experiment selected with ampicillin, and rejected the control.

The table, read closely

Three things stand out.

Astra and GPT-6.1 Sol are not clearly separated. Their confidence intervals overlap across most of their range (60.3–68.4%). OpenAI's own error bars, computed by resampling whole tasks, don't support calling Astra the better wet-lab reader; they support saying the GPT-6 generation is far ahead of GPT-5.x. OpenAI frames the comparison as Astra versus GPT-5.6 Sol, which is where the gap is unambiguous.

The better models write less. Astra uses about half the output tokens of GPT-5.6 Sol (4,704 vs 9,099) for 23 more points, and GPT-5.2 spends 27,620 tokens, almost six times Astra's, to reach 31.2%. On this kind of task, more thinking from a weaker model doesn't buy its way up the table. OpenAI's own explanation is that "stronger native multimodal capabilities may be key", which fits: most of these tasks fail or succeed at the moment of looking at an image.

Price changes the ranking. Using OpenAI's list prices for output tokens (Astra $50 per million, GPT-6.1 Sol $10, GPT-5.6 Sol $20, GPT-6 Luna $0.50), the output side of an xhigh run costs about $0.24 per task for Astra, $0.05 for GPT-6.1 Sol, $0.18 for GPT-5.6 Sol and under a cent for Luna. That is our arithmetic, and it leaves out input tokens, which for gels, microscopy stacks and videos may well dominate; OpenAI doesn't report them. Even so, GPT-6.1 Sol reaches a statistically similar score for roughly a fifth of Astra's output bill, and beats GPT-5.6 Sol by 18 points while costing less. Luna, the cheap model, lands below GPT-5.6 Sol and spends the most tokens of any GPT-6 model doing it.

Where Astra still gets it wrong

OpenAI's write-up is unusually specific about failures, and they are the useful part for anyone thinking of putting an agent near a bench:

• Assumption over controls. In a colony-PCR task Astra grouped the bands correctly, then assumed the larger product was the successful clone and picked lanes 4, 6, 7 and 8. The controls pointed to 5, 9 and 10. Following its answer would mean purifying the wrong plasmid.
• Discarding contradicting evidence. Asked about vacuoles, Astra noted that contrast varies with focus, inspected cells where vacuoles looked lighter, and still concluded they are generally darker.
• Counting. On a colony plate its marks merged close neighbours, missed visible colonies and counted empty agar, for an overcount of about 12%, which inflates a transformation-efficiency estimate by the same proportion.

None of these is exotic. They are the mistakes a careful junior researcher learns to avoid, which is rather the point of the benchmark.

What we can't check

The tasks, rubrics, raw results and evaluation code are not public, and OpenAI's post links no paper or dataset. LAB-Bench released about 80% of its questions and held back 20% to watch for contamination; BioBasicsBench, for now, can only be run by OpenAI. The aggregator BenchLM lists the scores as display-only for that reason.

The model list is all OpenAI: seven models across 37 model-and-effort settings, five attempts per task, 18,500 graded results (37 × 100 × 5, which checks out). There is no Claude, Gemini or open-weight model, so the benchmark says nothing yet about how OpenAI compares with anyone else here, including Anthropic, whose lab recently used Claude to solve a phage vault structure.

One model is missing that you might expect in a biology evaluation: GPT-Rosalind-6, OpenAI's gated life-sciences model. Its absence matters less than it seems. As we reported from its system card addendum, Rosalind-6 uses the same weights as GPT-6 Astra with no extra training; the difference is refusal policy and access. Astra's 66.7% is, in effect, the score of Rosalind's weights too, unless refusals depress Astra's result on some tasks. OpenAI doesn't break out refusals for BioBasicsBench.

OpenAI ties the benchmark to "biosafety, and biosecurity" as well as research speed, and that is the honest framing. A model that reads raw lab data well is more useful to a scientist and more capable in the hands of someone less careful. A benchmark that only its maker can run tells us the first part is improving quickly. It doesn't yet let anyone else measure either.