The AI security community has created 195 benchmarks since 2018, yet the fundamental problem isn’t scarcity. It’s fragmentation so severe the benchmarks are barely comparable.
Research cataloging the ecosystem found no shared language for measurement. “Standard” metrics like accuracy mask materially different judges, aggregation rules, and threat models. Of the 195: 137 benchmarks have abandoned repositories, 96 stale Hugging Face datasets, 165 evaluate only English. There’s no stewardship norm for post-publication maintenance and no principled way to choose which benchmark actually tells you something useful.
This is a classic problem in science: without shared definitions, abundance becomes a liability. Dark Reading coverage frames it as a benchmarking challenge; it’s really a governance one. The field needs standardization and maintenance norms far more than another benchmark. NIST’s benchmark guidance points toward that, emphasizing statistical rigor and reproducibility rather than quantity.
More benchmarks won’t fix fragmentation. Stewardship will.
Coverage: So Many AI Security Benchmarks, But None Tell the Whole Story
Related on this blog: OpenAI Tripled Its ARC-AGI-3 Score by Fixing Its Own Plumbing • Typed Decision Models: The Roster and the Benchmark Fine Print • [AUTO] Anthropic’s Evaluation Blind Spot