Gemini 4 Carbon takes Artificial Analysis' top spot by a point, and wins its knowledge test by guessing less

Google's Gemini 4 Carbon launched on October 28 with a claim built on someone else's leaderboard: first place on version 5 of the Artificial Analysis Intelligence Index, at 59. About twenty minutes after the announcement, Artificial Analysis posted its own numbers, and the claim holds. Carbon (high reasoning) scores 59.4 to Claude Opus 5.5 (max) at 58.4. GPT-6 Astra (max) is at 56.9 and GPT-6.1 Sol (max) at 54.5.

That is a one-point lead, on a model hardly anyone can use yet. Google is giving Carbon to "a subset of cyber defenders" first and then to partners in its Fairwind Program. Artificial Analysis tested it through Google's own API and labels it on the model page with "No publicly available API". So the ranking is independent, but nobody outside that circle can rerun any of it.

A narrow lead built from mixed results

The breakdown on Artificial Analysis' model page shows a model that is clearly strong but doesn't win across the board. Here are the per-evaluation scores for the top three (AA-Briefcase, the ninth evaluation, is an Elo rating and is discussed below):

Carbon comes first on four of the nine evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience and AutomationBench-AA. On AutomationBench the runner-up is its own predecessor, Gemini 4 Argon, 0.3 points behind. On Terminal-Bench 4.0 it is second, behind Kimi K3.1 Pro at 67.2%. FrontierCode 2.0, Cognition's private benchmark that grades whether a maintainer would merge the change, is the one where Carbon trails Opus 5.5 by almost six points. On AA-Briefcase, the agentic knowledge-work test, Carbon's 1609 Elo sits behind Grok 4.7, Muse Wonder and Kimi K3.1 Pro, according to Artificial Analysis' thread.

Artificial Analysis' separate Coding Agent Index, which runs each model in its maker's own agent, puts Carbon in Antigravity CLI at 59, three points behind Claude Code with Opus 5.5 (62). Argon scored 53, so the step up is real even without first place.

Winning a knowledge test by guessing less

The most interesting row is AA-Omniscience, where Carbon's margin over Opus 5.5 is one of its widest. The score is built to punish guessing: a correct answer adds to it, a wrong answer subtracts, and saying "I don't know" costs nothing.

Under those rules, Carbon wins while knowing less. It answers 58.3% of questions correctly. Opus 5.5 (max) gets 65.5%, and GPT-6.1 Sol (max) and GPT-6 Astra (max) both land around 62%. What separates them is what happens on the rest. Carbon's hallucination rate, the share of non-correct answers that are confidently wrong rather than abstentions, is 11.4%. Opus 5.5 is at 50.3%. Put back into whole-test terms, Carbon is wrong on about 5% of all questions and Opus on about 17%, and that gap is enough to flip a seven-point accuracy deficit into a five-point lead. The arithmetic checks out exactly against the published scores: 58.3 minus 4.8 is 53.5, and 65.5 minus 17.4 is 48.1.

That isn't a flaw in the benchmark. Refusing to make things up is the behavior Omniscience is meant to reward, and it fits Google's pitch that Carbon "cuts unsupported claims by approximately 2x over Argon". The two measurements don't line up neatly, though. On Omniscience, Carbon's hallucination rate is higher than Argon's (11% against 4%), because it now attempts more questions. That is still the second-lowest rate among the 78 model configurations on Artificial Analysis' comparison, after Argon. Google's 2x figure comes from internal testing of agentic deliverables that nobody outside can inspect.

The price, and what can't be checked

Artificial Analysis costs Carbon at its eventual list price of $4 per million input tokens and $20 per million output tokens, which gives $7.76 per index task. That is about 23% less than Opus 5.5 (max) at $10.04 and roughly 1.2 times GPT-6 Astra (max). GPT-6.1 Sol (max) reaches 54.5 for $1.27. Google's introductory price is half the list price, and Artificial Analysis notes there is no confirmed end date. The page also shows a 97-second wait for the first token on Google's API. For a model in limited rollout with a single provider, that figure may say more about the endpoint than about the model.

Everything else in the launch post is Google's own measurement. That includes the cybersecurity numbers that justify the gated release: 88.2% on SEC-bench Pro, 54.8% on ExploitGym and 71.0% on CWE-bench. Google says these were "evaluated without guardrails", which describes a configuration the defenders getting access presumably won't use. The same goes for 0.0% successful cheating on Terminal-Bench 4.0, an almost 80% drop in high-severity misalignment alerts, and a 0.0% attack success rate on Gray Swan's prompt-injection benchmark. As of publication there is no model card or safety report for Carbon. For Argon, Google pointed to a Safety and Security report. The WebDev Arena placing (fourth, at 1768) we could not check, because the arena site blocks our reader.

So the top of the leaderboard now belongs to a model you can't buy, by one point, with its biggest knowledge win coming from declining to answer. None of that changes the fact that, by Artificial Analysis' independent numbers, Carbon is the strongest general model it has measured. The cyber claims that set the terms of its release are the part that still rests on Google's word.