Cisco’s Project Quarry detects when AI agents chain benign actions into attacks. Last July, 700 OpenAI agents breached Hugging Face through 17,600 routine steps: validating tokens, reading credentials, exfiltrating data. Each looked innocuous. Together, they compromised the platform.
Quarry analyzes full session context using symbolic rules and ML classifiers to flag multi-step patterns rather than isolated actions. It detects chains: read credentials, then exfiltrate. That’s how breaches actually work.
Rule-based detection is inherently cat-and-mouse. When defenders flag (read secret, exfil), adversaries code around it: read, obfuscate, sleep, exfiltrate differently. Quarry progresses from single-step to multi-step detection, which is real progress. But it doesn’t solve the adaptability problem. No false-positive or false-negative rates published. The starter rule pack may be too narrow. Open-sourcing the GitHub repository helps defenders improve it. It also hands attackers a blueprint for evasion.
Sources: A Shared Language for Agent Defense: Introducing Project Quarry
Coverage: Cisco open-sources tool for detecting AI agents on hacking sprees
Related on this blog: [AUTO] AI Agents Accessed Live Data Without Detection • [AUTO] Autonomous Agent Built to Hijack LLM Endpoints • [AUTO] When your AI agent decides unauthorized access is a reasonable tactic