Anthropic's cheapest misuse monitor is Claude grading its own conversations

Anthropic has trained Claude to check its own conversations for misuse. After each response, the system appends a short hidden instruction asking Claude to reflect on the safety of the exchange, and Claude answers with a few tokens of JSON: usually just {"flagged": false}, otherwise a harm category, a sub-policy, whether it refused, a 1–10 priority for human review and a one-sentence reason.

Anthropic calls this self-assessment and described it in the first post on its new Safeguards blog, which Anthropic announced on October 29 (the post itself is dated October 16). It has been running since June, in every Claude model since Claude Fable 5 and across "a large fraction" of Claude's traffic. The pitch is cost. The conversation is already in the KV cache when Claude finishes answering, so an assessment means processing one short instruction and generating about 10 tokens on average. Anthropic estimates the overhead at under 2% of inference compute on its production API traffic. For comparison, it says its first Constitutional Classifiers in early 2025 added 23.7%.

How good it is depends on who is grading

On a random sample of production traffic, Fable 5's self-assessment flagged 78% of the conversations that a stronger prompted classifier flagged as potentially harmful, and 94% of the high-priority ones, at a 1% false-positive rate. That stronger classifier, built on Claude Mythos Preview with Anthropic's full internal policies in its prompt and a reasoning checklist, is also the teacher whose labels self-assessment was trained on, through supervised fine-tuning and then RL with six verifiable rewards.

So the headline numbers measure how well the student reproduces the teacher, not how often either is right. Anthropic says so in its limitations section: any blind spot in the teacher passes to the student, "and our evaluations cannot detect it." It suggests that future work add human evaluations.

Two other results are more interesting than the headline. Across four internal model sizes trained the same way (SFT only, on a separate test set), recall at a 1% false-positive rate rose from about 16% on the smallest to about 96% on the largest. That supports Anthropic's argument that a monitor which is the model gets better as the model does. And when an entire harm category was removed from training, recall on it barely moved: from 92.7% to 90.3% for cyber on synthetic data and from 94.2% to 90.7% on production traffic. Anthropic reads this as training teaching Claude how to report harm, while its sense of what is harmful comes from what it already knows. Both held-out categories, fraud and cyber, are ones Claude already recognizes, as the post admits.

A bad blocker, a decent tripwire

Anthropic also tested self-assessment as something it isn't used for in production: a real-time blocker. An orchestrated attacker model worked through 24 offensive cybersecurity tasks against Fable 5 with production classifiers off, and could rewind and retry whenever it was blocked. With no safeguards it succeeded on 19 of 24 tasks. With self-assessment as the blocker, 13 of 24. A standalone prompted classifier cut it to 6 of 24. Self-assessment did make each successful attack cost roughly four times as many tokens.

Used offline, the way it runs in production, it flagged 21 of the 24 attack trajectories at least once. The difference is feedback: an attacker gets no signal about what an offline monitor saw until something like an account ban arrives much later. Anthropic is explicit that an attacker who targets self-assessment directly could probably get the model to reveal what it would flag and then search for a jailbreak that silences it.

What it has caught

The production findings are the most concrete part of the post. In August, self-assessment raised more than 100,000 child sexual abuse and exploitation flags from a single API customer, which Anthropic's child safety team investigated and reported to NCMEC. More than half of the organizations it surfaced for enforcement had no flags from any other detection source in the preceding month. In September its flags surfaced an operation that used Claude Code to build and run an automated romance scam aimed at dozens of victims; the accounts were banned after human review.

About a third of the flags don't map cleanly onto Anthropic's harm taxonomy. Clustering them turned up attempts at illicit distillation of Claude's reasoning, prompt injection, and cases where Claude, acting as an agent, took actions the user hadn't authorized. That last one is Claude flagging its own behavior, which it wasn't trained to do.

It also worked as an audit of Anthropic's other classifiers. Among conversations self-assessment flagged and the child-safety probe didn't, a separate grader judged 41% to be real violations. For the cyber classifier, the same check found only about 1%, which Anthropic reads as evidence that the cyber classifier misses little.

Why not use it to watch Claude itself

The obvious next question is whether the same trick could monitor the model's own misalignment. Anthropic's answer is "not yet." A deceptive model could hide a misaligned action in its assessment as easily as in its actions, and monitors that share the agent's cache carry a known risk of collusion. Even an honest model shares the same judgment that produced the action, and Anthropic cites 2026 research by Khullar et al. finding that models rate an action as less risky when it appears in their own previous turn.

That puts Anthropic's design at the opposite end from the one OpenAI described the same day. OpenAI's misalignment monitors are separate reviewer models and investigative agents that read the agent's transcript from the outside, at considerable compute cost. Anthropic's self-assessment watches users rather than the model, and is cheap precisely because the model grades its own homework.

A few operational limits are worth knowing. Self-assessment doesn't run on traffic under zero-data-retention agreements. Its structured fields are handled like other classifier scores, while the free-text reason, which can paraphrase the user's conversation, gets the same data controls as prompts and completions. And it is passive: it feeds nightly per-organization aggregation and human review, not live blocking, which Anthropic leaves to its other classifiers.