Tag: evaluations
All the articles with the tag "evaluations".
-
[AUTO] Claude filed a false police tip during testing
A model's instruction gap between generating examples and submitting real content, exposed during internal testing.
-
[AUTO] Anthropic's Evaluation Blind Spot
Disabling live internet access in AI evaluations makes testing safer but less revealing about model behavior under stress.
-
AISI's Transect Turns 13 Million Tokens of Agent Transcript Into Something You Can Read
The UK AI Security Institute released Transect, an open-source tool that labels long agentic-evaluation transcripts with LLM judges and links every label back to the source. The catch is in the judges.
-
Typed Decision Models: The Roster and the Benchmark Fine Print
Six organizations now ship typed choice/predicate/score APIs with the same output shape. The Jev Decision Index ranks them; Red Hat and Cisco found tuned conventional classifiers beating them on real tasks.