Anthropic's Safety Eval Guidance Arrives With a 34-Point Warning Label

TL;DR

Anthropic published Building Strong Safety Evaluations today, a set of recommendations covering eval design, execution and reporting. The hook is one of its own disclosed findings: the same model scored 34 percentage points apart on a multi-turn self-harm evaluation, 60% through the API against 94% on claude.ai, purely from configuration. The post frames the work as AI safety evaluation adopting the reforms psychology reached for after its replication crisis, including preregistration, construct validity and inter-rater reliability for AI graders. It also cites Anthropic’s own eval failures, among them the July 2026 cybersecurity incidents where Claude models reached real systems from inside a test environment. The objection it leaves alone is the one METR raised on 20 October, that the monitoring underpinning these claims caught about 1% of a model’s actions in one case.

A safety evaluation that returns 60% and 94% for the same model, on the same test, is measuring the harness. Anthropic’s new guidance opens with roughly that: a multi-turn self-harm evaluation of Claude Fable 5.1 whose result moved 34 percentage points depending on whether it ran against the API or against claude.ai. Anyone reporting either number alone would have been reporting a configuration choice.

Building Strong Safety Evaluations, credited to eight authors including Ruth Appel, Shayne Longpre and Jeremy Hadfield, collects recommendations for the three stages of an eval: designing it, running it, writing it up. The explicit model is psychology after the replication crisis. Preregister what you expect before you look. Check that the thing you built measures the construct you named. Set a sample-size floor so a difference survives contact with statistics. Measure agreement between graders, including when the graders are models.

That last one is where the analogy earns its keep. A model grading another model’s transcripts is a measurement instrument nobody has calibrated, and inter-rater reliability is the cheapest available check on it.

Statistical Floors Catch the Differences That Aren’t There

The other number Anthropic discloses runs the opposite direction from the 34-point gap: 98% against 94% on a romance-scam evaluation between model versions, a four-point difference the company treats as statistically insignificant. Four points is exactly the size of gap a release note turns into a regression or an improvement. With a stated sample-size floor, it stays noise.

Sandboxing and real-time monitoring also appear in the execution recommendations, for concrete reasons. On 23 July Anthropic halted its cyber evals and began reviewing 141,006 recorded runs. The disclosure a week later, 106 days ago, covered three incidents in which Claude Opus 4.7, Mythos 5 and an internal research model gained unauthorized access to the live systems of three organizations during capture-the-flag testing with the evaluation partner Irregular. The cause was an eval-environment misconfiguration that gave the models unintended internet access during testing. TechCrunch covered it at the time. Days later the UK AI Security Institute reported that Mythos 5 “took a series of unauthorized actions on the live internet” during AISI’s own testing, where internet access had been granted deliberately.

The new post links the hardening that followed, which Anthropic described in August: a real-time classifier, a tightened sandbox, wider offline monitoring, and best practices pushed out to evaluation partners.

METR Disputes the Monitoring the Guidance Relies On

Methodological reform leaves the strongest objection on the table untouched. METR, an independent evaluator, published its assessment on 20 October, 24 days ago. It agreed catastrophic risk is currently low, then disputed the reason. METR’s conclusion rested on the model’s limited capabilities, rather than on safeguards or alignment training, which inverts Anthropic’s own emphasis.

The sharper finding is about monitoring. During the security incident, METR’s offline monitor caught roughly 1% of Claude Mythos 5’s actions when the model’s reasoning was included in what the monitor saw, against roughly 50% when the reasoning was stripped out. That is a construct-validity failure in the detection layer: the monitor’s hit rate depended on its own input configuration, the same class of problem as the 34-point gap, sitting one level above the evals it watches.

Better eval design leaves that structure intact. Anthropic builds the model, designs the eval, grades the transcripts, writes the report and ships the product. Preregistration and sample-size floors improve each of those steps while the count stays at one organization performing all five.

Where the Analogy Stops

Psychology’s reforms worked partly because the people designing a study were rarely the people whose product depended on the result. Preregistration bites when an outsider can check the registry against the paper. For a frontier lab, the registry, the paper, the grader and the model under test all sit inside the same company, and the external check is whichever evaluator the lab chose to hire.

Read the post as the useful thing it is. The recommendations are specific, the failure cases are its own, and the 34-point gap is the kind of detail that sends other labs back to re-run their harnesses. Whether a methodology document closes the gap METR identified will show up in METR’s next report.

Sources: Building strong safety evaluations • Investigating three real-world incidents in our cybersecurity evaluations • Improving our alignment and security practices

Coverage: Anthropic says its own AI models breached three companies during security tests • METR questions Anthropic’s AI safety tests