LLM vulnerability auditors defer more to false assurances than false alarms
Original title: Trusting the Tool: Asymmetric Deference of LLM Vulnerability Auditors to Static-Analysis Evidence
Across 16,572 audits of PrimeVul pairs, true static-analysis notes barely helped, while fabricated assurances cut real-bug detection by up to 19 points.
Researchers built an intraprocedural C/C++ analyser and audited vulnerable and patched PrimeVul functions under six evidence conditions, including fabricated assurances and alarms. The study used 916 pairs, eleven untuned model configurations from four vendors, and 16,572 audits. True evidence raised accuracy by only 0.02. Relative to true notes, false assurances lowered detection of real vulnerabilities by up to 19 points; false alarms did not significantly lower accuracy and were disputed more often. A replication on seven new models found mean asymmetry of +0.16. Separate claim verification raised accuracy under false assurances by 12 points in a pre-specified test.
Unconfirmed: The second model's accuracy gain from claim verification was smaller and not statistically significant.
- True evidence improved accuracy by only +0.02.
- False assurances lowered real-vulnerability detection by up to 19 points.
- Replication mean asymmetry was +0.16, with a 95% CI of +0.10 to +0.24.
- Prompt instructions did not reliably reduce deference and the best prompt failed to replicate.
- Verifying claims in a separate call gained 12 points under false assurances (p=0.002).
From the original article
1 min read at arxiv.orgPipelines that combine large language models (LLMs) with static analysis let the model consume the analyser's output as evidence, but whether the model distinguishes true from false evidence, and whether its scrutiny depends on what the evidence claims, has not been measured.
By ZhiHong Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.