Where Do the Tokens Go? Understanding and Reducing Costs in LLM Agents for Vulnerability Discovery
A 200-trace CyberGym study finds reasoning dominates token cost, and AVRI lowers it.
Researchers open-coded 200 CyberGym traces from Codex, OpenCode, Cybench, and EnIGMA under an unaided baseline and four existing efficiency methods. Agents vary widely in success and cost, and higher spending does not consistently improve outcomes. Code localization, vulnerability reasoning, and trigger design account for 60.4% of tokens, while only 24.4% of matched comparisons preserve success at lower cost. Their AVRI interface, using a persistent Bidirectional Evidence Trace, reduces cost 18.0% for Codex and 23.7% for OpenCode on 20 tasks without lowering success.
50