LLM agent detector evaluations hide cross-benchmark issues
Original title: What Standard Evaluations Hide About Detectors of Hijacked LLM Agents: Rare Attacks, Unseen Tools and Benchmark Transfer
Detectors trained on one benchmark fail on another with AUROC as low as 0.47.
Eleven audited evaluations of trace-based detectors for hijacked LLM agents contain 16-86% attacks or anomalies. On AgentDrift and AgentDojo benchmarks, detectors with AUROC 0.879 reach only 0.076 precision at 2% attack rate. Performance reverses with unseen tools below 50% coverage; dropout mitigates reversal. No AgentDrift-trained detector exceeds chance on AgentDojo.
- 11 audited evaluations with 16-86% attack samples
- AUROC 0.879 reaches 0.076 precision at 2% attack rate
- Performance reverses below 50% tool coverage
- No cross-benchmark transfer (AUROC 0.47-0.52)
- AlignmentCheck monitor reaches 0.930 but high false alerts
From the original article
1 min read at arxiv.orgDetectors can flag hijacked large language model (LLM) agents from the traces of the tools they call. In the eleven evaluations we audited, attacks or anomalies make up 16% to 86% of the test samples, no evaluation varies that share, and six report no test on data from a second benchmark or source. We ask what such evaluations hide.
By Mahmoud Abbasi · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.