VulnSAGE checks LLM judgments of SAST reports against program structure
Original title: VulnSAGE: Adjudicating SAST Reports by Connecting LLM Reasoning to Program Structure
On 187 labeled reports it rejected 89 of 97 false alarms and kept 85 of 90 real defects.
A paper introduces VulnSAGE, a system that adjudicates static application security testing reports by tying large language model reasoning to a shared program graph. A report-centered slicer recovers dependencies of the flagged operation, and CLAIM records each interpretation as a checkable semantic fact. Program analysis verifies those premises before a verdict. Across 187 labeled reports from four corpora and five vulnerability families, it rejected 89 of 97 false alarms and retained 85 of 90 real defects. On 45,440 alerts from five tools over 200 C/C++ GitHub projects, it reproduced 104 defects; maintainers confirmed 24 of 26 responses, including ten previously unknown defects.
Unconfirmed: Defect counts and maintainer confirmations are reported by the paper; individual CVE identifiers are not listed.
- Under one PoC procedure, 25 positive verdicts reproduced, versus three for LLM4SA.
- Nine of the ten previously unknown confirmed defects are already fixed.
- Evaluation covered five vulnerability families in four labeled corpora.
- Alerts came from five SAST tools on the 200 most-starred C/C++ GitHub projects.
- Products
- VulnSAGE
From the original article
1 min read at arxiv.orgStatic Application Security Testing (SAST) reports flag operations whose safety depends on project-specific context, and maintainers must decide which reports describe real defects. Large language models (LLMs) can interpret this context, but adjudicating a report with them requires the decisive code, which often lies far from the reported line, and a decision basis that program analysis can check.
By Jinbao Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.