DeepMind-backed AI-text detector explains its verdicts, and the explanations help rewrite text to evade it

AI-text detectors are used on student essays and conference reviews, and they usually can't say why they flagged a document. A paper accepted at EMNLP 2026, from the University of Pennsylvania and Google DeepMind, builds detectors that can. Its own experiments also show where that leads. Once a detector can list the features it relies on, a language model can be told to remove them, and the detector stops catching it.

How the detector works

The authors run text through Gemma-2-2B and decompose its internal activations with sparse autoencoders, using DeepMind's GemmaScope set. Each autoencoder feature gets a plain-language label written by a second model (Gemini 3.1 Flash-Lite), and features whose labels describe their behavior poorly are masked out. A simple logistic regression over the remaining features makes the call. That way every prediction comes with a ranked list of features such as "the word 'thorough'" or "first person singular pronoun I at start of sentences or after punctuation", each with a weight. Code and trained models are on GitHub under Apache 2.0.

Interpretability didn't cost accuracy, at least on these tests

The detectors were trained on the English portion of the COLING 2025 shared-task data (610,000 documents from 41 language models) and tested on held-out COLING splits and the RAID benchmark. For two of the three variants, training the classifier on autoencoder features generalized better than fine-tuning the model directly. On the COLING test split, the fine-tuned Gemma detector reaches an F1 of 75.2. The best feature-based variant reaches 92.2, with or without the interpretability mask. Binoculars, a popular zero-shot detector, gets 81.4 there. On RAID the margins are smaller: 91.8 against Binoculars' 87.6.

These are the authors' tests on established datasets. Detection rates are reported at a 5% false-positive rate, meaning one human document in twenty is wrongly flagged. That is a high rate for any setting where a flag has consequences. The paper doesn't measure false positives on groups known to trip detectors, such as non-native English writers.

What the features reveal

The second dataset is the Peer Review Detection Benchmark: 106,413 human reviews from NeurIPS and ICLR (2016 to 2024), each paired with AI-generated reviews. Here every classifier, including the baselines, scores close to 100. The authors don't treat that as a triumph, and their feature lists show why. One of the strongest GPT-4o signals is a feature labeled "double colons following bolded markdown section headers". It appears in 11 human reviews and more than 54,000 GPT-4o reviews. A strong Llama signal, an opening square bracket in lists, appears in 7 human reviews against more than 22,000 Llama ones. On the human side the detector leans on style: sentences starting with "I", hedges like "seems", and "see" used to point elsewhere. On COLING, the authors report, the base-model features pick up on domain residue such as HTML attributes and license boilerplate.

So the detector is largely catching how a particular model formats its output, not anything fundamental about machine writing. The authors say as much: the features "may fail to generalize to other domains and models". Being able to see that is the point of the method, and it's something a black-box detector would never show.

The same list works as an evasion guide

The paper then tests the obvious attack. Gemini rewrote 2,500 AI-generated documents per dataset using five prompts. Generic humanizing prompts flipped at most 8.7% of the detector's verdicts, and often none. Giving the rewriter the detector's own list of active features and telling it to remove them worked much better, and repeating that three times worked best.

The main text summarizes the three-pass attack as succeeding "60-100% across all conditions". The paper's own appendix table doesn't support that range. Of the nine detector-and-dataset combinations, five come in below 60%. Against the variant built on fine-tuned autoencoders, the attack succeeded on only 19.7% of GPT-4o reviews and 21.8% of Llama reviews. The appendix explains this: that detector's evidence is spread over about 150 densely firing features, which no ten-item list captures. On COLING the three-pass attack got 52.6% to 63.2%. Its best result, 97.1%, came on Llama reviews against the detector built on the fine-tuned model.

The edits also weren't as light as instructed. Every prompt told the rewriter to change under 30% of the text. After three passes, measured word-level edits reached as much as 38%. A defender could also note that AUROC holds up much better than the success rate (for one detector on COLING it falls from 0.96 to 0.82), because the attack only has to push documents just past a fixed threshold.

The trade-off

The paper is honest about what auditable features reveal: on peer reviews, a detector that scores near 100 turns out to be reading markdown habits. That is useful for anyone deciding whether to trust a detector. But the same transparency, given to the person being checked, turns into a short list of things to edit. Showing the reasoning and resisting evasion pull against each other, and the paper's data suggests the most robust variant is the one whose evidence is too spread out to fit in a short explanation. Anyone deploying such a tool will have to choose how much of that explanation to show, and to whom.