Where Did the Repair First Go Wrong? Localizing the Origins of Silent Failures in Agentic Vulnerability Repair
SAGE traces where coding agents first abandon security while producing patches that still pass functional tests.
The paper introduces SAGE, a trace-based method that scores security reasoning at each turn and reconstructs code history to find the earliest divergence from a repair's security intent. On 95 confirmed silent failures from 3,684 traces by six agent frameworks and six base models on SecurityEval and CVEfixes, SAGE assigned an origin in 93 cases. Most origins were an unmet security requirement or a weak defense, and only five were the code edit itself. When the agent wrote the vulnerable code, the origin came before that write in 14 of 19 cases.
- Silent failures are patches that pass checks but remain vulnerable.
- SAGE localized an origin in 93 of 95 confirmed silent failures.
- Most failures start as missed requirements or weak defenses, not the edit.
- Origin type was more reproducible than the exact turn.
From the original article
1 min read at arxiv.orgLocalizing where an LLM-based agent first fails to uphold security during a repair can show which stage of its workflow needs an additional safeguard. This is difficult for silent failures, which are patches that pass syntactic and functional checks but still contain a security vulnerability.
By Wenji Bai · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.