Anthropic's safety-eval guide: Claude Fable 5.1 scored 60% on a self-harm eval via the API and 94% on claude.ai

Anthropic has published "Building strong safety evaluations," a guide on its Engineering blog (November 13) with ten recommendations for designing, running and reporting AI safety evals, written by Ruth Appel, Evan Frondorf, Jeremy Hadfield, Rebecca Hiscott, Kamya Jagadish, Shayne Longpre, Madeleine Tucker and Brett Wittmershaus. Most of the advice is borrowed, openly, from psychology's replication-crisis reforms: preregister your analysis, validate your grader against human labels, report confidence intervals. The more interesting parts are the numbers Anthropic uses as examples, because several of them say more about how Claude's safety scores are produced than its system cards usually spell out.

The system prompt is doing a lot of the work

The clearest one: on its multi-turn safety evals, Claude Fable 5.1's appropriate-response rate differed by "as high as 34 percentage points" between the raw model on the API and claude.ai, where Anthropic's system prompt is applied.

The guide also gives a case of that kind. Reviewing suicide and self-harm transcripts before Fable 5.1's release, Anthropic found Claude "sometimes implicitly validated self-harm as a coping strategy," even in replies that discouraged it. The fix was a line in the claude.ai system prompt, "Claude does not tell someone that self-harm works, helps, or does something for them, even when they say so themselves," plus other targeted prompt changes. The result: 60% on the API against 94% on claude.ai with the new prompt. That 34-point gap is the same size as the maximum Anthropic cites, though the post doesn't say it is the same measurement.

For anyone building on the Claude API, this is the practical takeaway. The safety figures people quote for claude.ai include a system prompt you don't get. An application with its own prompt, talking to the raw model, is closer to the API configuration than to claude.ai's, and Anthropic's own advice is to "match the eval setup to the deployment you want to test." Anthropic's system cards do report API and claude.ai results separately; the guide is a reminder to read the right column.

Single-turn evals have stopped telling models apart

Anthropic says the aggregate harmless-response rates of its single-turn evaluations sit between 94% and 98% "for every recent model" on the API, and calls them saturated, "useful mainly as regression checks on known failure modes." Those suites are not small, either: 16 policy areas in seven languages across several product surfaces. When a company says a new model is safer than the last one, the single-turn numbers are now unlikely to be where the difference shows up; the multi-turn evals, where scores spread much more widely, are.

98% versus 94% is a tie

The guide's statistics section comes with an honest example. On a multi-turn romance-scam eval on claude.ai, Claude Opus 5 scored 98% and Opus 5.5 94%, and the difference "wasn't statistically significant." Anthropic also notes that confidence intervals on multi-turn API evals can exceed ±10 percentage points.

The post doesn't give the size of that eval, but it does give the floor it uses: 100 items for a complex multi-turn eval. We checked what that floor buys. With 100 conversations per model, 98% against 94% gives p ≈ 0.28 in a Fisher exact test, nowhere near significance. The gap first crosses p < 0.05 at around 300 conversations per model, and detecting it reliably (80% power, unpaired) takes about 375 per model by the standard sample-size formula. Paired comparisons on the same items, which the guide recommends, would need fewer, but the order of magnitude stands: a four-point difference near the ceiling needs several times Anthropic's own minimum before it means anything. That is worth remembering the next time a one- or two-point safety improvement appears in a launch post, Anthropic's or anyone else's.

The rest of the list

The other recommendations are sound and mostly familiar: measure appropriate responses rather than refusals (a model that refuses every self-harm prompt gets a perfect refusal score and is not safer); pair harmful prompts with benign ones in the same domain so over-refusal is counted; make grader criteria atomic yes/no questions; keep design metadata away from the grader; check graders against human labels using chance-corrected agreement such as Cohen's kappa; and have broken grades fail loudly instead of being silently scored. Anthropic describes an internal tool in which domain experts write a scenario specification, the first user turn is fixed, a separate Claude model plays the user, and a grader model scores each transcript; the public counterpart it points to is its Petri toolkit, built on the UK AI Security Institute's Inspect framework.

The guide also says, briefly, that after "our own incidents" Anthropic hardened its evaluation environments, temporarily stopped running many evals that give models internet access, and added real-time monitoring to web-search evals that still need it. It doesn't say more than that.

If you take three things from it, Anthropic suggests these: define the construct before collecting data, validate the grader against humans, and report uncertainty. Its own examples show why the third one matters most when reading other people's numbers.