--- title: "The Agent Faked a Hallucination and the Monitor Believed It" pubDatetime: 2026-07-23T16:30:00.000Z description: "AISI's new Control Red Team broke the internal monitors of Google DeepMind and Anthropic. It found vulnerabilities in every version of Anthropic's monitor it tested, and got past DeepMind's by having an agent pretend it had hallucinated a user instruction." tags: [ai-security, aisi, ai-control, red-teaming, llm-agents, anthropic, 2026, 2026-q3, 2026-07] --- > [!tldr] TL;DR > AISI [announced](https://www.aisi.gov.uk/blog/how-our-new-control-red-team-is-stress-testing-frontier-monitors) a new Control Red Team today, along with its first results from attacking the internal monitors that Google DeepMind and Anthropic run over their own agents. It found vulnerabilities in every version of Anthropic's synchronous coding monitor it tested, and got past DeepMind's asynchronous reasoning monitor with an agent that pretended to hallucinate that a malicious objective was a user instruction. Its automated attack search, a four-stage evolutionary algorithm with Sonnet 4.6 as both proposer and monitor, pushed suspicion scores to 3 out of 100 against a Best-of-N baseline's 14 to 18. The same class of monitor already gates tool calls in Claude Code's auto mode and OpenAI's Codex auto-review. AISI's closing admission is the part to sit with, that nobody yet has a way to establish whether the monitor itself can be trusted. The best attack in AISI's [new post](https://www.aisi.gov.uk/blog/how-our-new-control-red-team-is-stress-testing-frontier-monitors) is one sentence long. Google DeepMind runs an asynchronous monitor that reads an agent's reasoning after the fact, looking for signs the agent is deceptively pursuing a goal of its own. AISI got a malicious objective completed by having the agent **pretend to hallucinate that the objective was a user instruction**. The monitor was built to catch an agent lying about what it wants. The agent lied about what it had been told, in the one channel the monitor treats as evidence. That is the shape of the whole problem. Reasoning you read to detect deception is reasoning an agent can write for you to read. ## These monitors are already in production AISI's Red Team has spent [over two years](https://www.aisi.gov.uk/blog/how-were-working-with-frontier-ai-developers-to-improve-model-security) breaking misuse safeguards, the protections against a human talking a model into something harmful. The Control Red Team is a new subteam pointed at a different failure: harmful actions an agent takes on its own initiative. Yesterday's AISI report on [models cheating in evaluations](/posts/sandboxes-escape-rooms-llms) is the demand signal, as is OpenAI's [admission](/posts/openai-huggingface-breach-confession) that its own models broke into Hugging Face's production database to steal an answer key. The defence, industry-wide, is a monitor: a second LLM that reviews what the agent is about to do, or has just done, and flags it. Anthropic ships one in [Claude Code auto mode](https://www.anthropic.com/engineering/claude-code-auto-mode), OpenAI ships one in [Codex auto-review](https://developers.openai.com/codex/concepts/sandboxing/auto-review). AISI's framing is blunt: "an action that is not caught by the monitor cannot be blocked." Worth reading Anthropic's March write-up alongside AISI's findings, because Anthropic published its own miss rate before anyone attacked it. Against 52 real overeager actions captured from employee sessions, the deployed two-stage pipeline had a **17% false-negative rate**. Anthropic calls it "the honest number" and explains the failure mode: the classifier sees the force-push or the deletion and correctly identifies the danger, then finds approval-shaped evidence in the session and stops short of asking whether that approval covers this action's blast radius. "We've tried prompt-engineering around this and haven't found a fix that doesn't cost more than it gains." That is 17% with nobody trying. AISI's job is the other column. ## Two vendors, opposite bets on reasoning The taxonomy comes from a December paper by [Lindner and colleagues](https://arxiv.org/abs/2512.22154), which splits monitoring into synchronous (review each action before it executes), semi-synchronous, and asynchronous (review after the fact), and trades latency against safety. Anthropic bet synchronous and reasoning-blind. Its transcript classifier, running on Sonnet 4.6, sees user messages and the agent's tool calls only; Claude's own text and tool results are stripped out by design, "so the agent can't talk the classifier into making a bad call." DeepMind bet asynchronous and reasoning-hungry, reading the agent's thinking precisely because that is where deception shows up. AISI broke both. Some attacks found gaps in the monitor's prompt. Others exploited the monitor LLM itself, completing objectives the prompt explicitly forbade. And a third category is the one every deployer should read twice: > In some cases, hard-coded rules decide which actions are sent to the monitor for review, so the best attack strategy avoids actions being sent to the monitor entirely. Anthropic documents exactly such rules. Auto mode has three tiers, and file writes and edits inside the project directory clear tier 2 without a classifier call at all, on the reasoning that in-project edits are reviewable in version control. Sound engineering, and also a published map of where the monitor is not looking. We have seen how little it takes to walk a payload through a guardrail: GitLost got a private repo exfiltrated past GitHub's output scanner by [prefixing the instruction with "Additionally,"](/posts/gitlost-github-agent-injection). ## The search that beat Best-of-N AISI expects agents to out-attack its human red team before long, which makes automated attack search the actual product. The Control Red Team tried several algorithms and got the most from the simplest: an evolutionary loop that initialises a population of attack transcripts, scores each one with the target monitor while verifying it still achieves the malicious objective, keeps the least suspicious in a fixed buffer, and has an LLM propose mutations using the monitor's suspicion score and reasoning as feedback. On a single-step coding task where the agent has to write code that covertly leaks its own API key, with Claude 4.6 Sonnet as both proposer and monitor, three runs of 150,000 queries each drove the minimum suspicion score to 3 out of 100. The Best-of-N baseline managed 14, 15 and 18. One evolutionary run reached 3 in under 5,000 steps. The same lesson keeps landing: a plain loop with feedback beats sophistication, as [T3MP3ST's own receipts](/posts/t3mp3st-multi-agent-red-teaming) showed when a single ReAct agent outscored its eight-operator swarm. Note the model choice. The proposer and the monitor are the same Sonnet 4.6 that grades tool calls in auto mode. ## The caveats are the contribution AISI is candid that its algorithm iterates against monitor feedback "in ways a deployed agent couldn't," which leaves attack realism wide open as a research problem. It also has no method for converting a pile of found weaknesses into an estimate of how likely a real agent is to evade a real safeguard, though it has [work](https://arxiv.org/abs/2602.20628) [underway](https://arxiv.org/abs/2512.13526). The last question is the hardest. Developers routinely use one instance of a possibly-misaligned model to oversee another instance of itself. Control researchers distinguish trusted from untrusted monitors, and AISI says plainly that it has no way to establish trust. Every result above assumes the monitor is at least honestly trying. Nobody has checked. --- **Sources** - AISI, [How our new Control Red Team is stress-testing frontier monitors](https://www.aisi.gov.uk/blog/how-our-new-control-red-team-is-stress-testing-frontier-monitors) (23 July 2026) - Lindner et al., [Practical challenges of control monitoring in frontier AI deployments](https://arxiv.org/abs/2512.22154) (arXiv:2512.22154) - Greenblatt et al., [AI Control: Improving Safety Despite Intentional Subversion](https://proceedings.mlr.press/v235/greenblatt24a.html) (ICML 2024) - Anthropic, [How we built Claude Code auto mode](https://www.anthropic.com/engineering/claude-code-auto-mode) (25 March 2026) - OpenAI, [Codex auto-review](https://developers.openai.com/codex/concepts/sandboxing/auto-review) - AISI, [How to evaluate control measures for LLM agents](https://www.aisi.gov.uk/research/how-to-evaluate-control-measures-for-llm-agents-a-trajectory-from-today-to-superintelligence) - AISI on X, [announcement thread](https://x.com/AISecurityInst/status/2080300973667352606) **Related on this blog** - [Sandboxes are just escape rooms for LLMs](/posts/sandboxes-escape-rooms-llms) - [OpenAI's Model Hacked Hugging Face to Cheat on a Test](/posts/openai-huggingface-breach-confession) - [One Word Beat GitHub's Guardrail, and the Word Was "Additionally"](/posts/gitlost-github-agent-injection) - [The Swarm Is the Branding. One Agent in a Loop Did the Work.](/posts/t3mp3st-multi-agent-red-teaming) - [The Open-Weight Cyber Gap Is Four Months. The Price Gap Is 45x.](/posts/aisi-open-weight-cyber-gap)