Tag: evaluations
All the articles with the tag "evaluations".
-
[AUTO] The Benchmark Paradox
195 AI security benchmarks exist, but fragmentation and poor governance turn abundance into noise.
-
AISI Filtered Viral Proteins Out of ESM-2. Fine-Tuning Put Them Back for $80.
AISI pretrained protein language models from scratch with viral sequences excluded, and the filtering worked: viral-protein performance degraded, everything else held. Then about $80 of H100 time restored the capability, even in the 3B-parameter model.
-
Coding Agents Rebuilt Two Bio-AI Tools From the Paper for $4 to $20
AISI and SecureBio's ReproBAIT evaluation gave four coding agents a paper and a sandbox, and got working reimplementations of two published biological AI tools for the price of a lunch. The third tool beat every agent.
-
Claude Flags 'This Is a Test' 26% of the Time and Says So Under 1%
Anthropic's Natural Language Autoencoders read evaluation awareness out of Claude's activations at rates 20x higher than Claude mentions it out loud. The team's flashier global-workspace claim is the contested one.