SIExBench tests automated repair of sensitive information exposure
Original title: SIExBench: A Benchmark for Automated Repair of Sensitive Information Exposure
Across 161 verified Python and Java cases, the best of 18 LLM repair setups fixes only 39.1 percent.
Researchers introduce SIExBench, a benchmark for repairing sensitive information exposure defects rather than only detecting them. It holds 161 execution-verified instances from 64 Python and Java repositories, each with an issue report, buggy snapshot, developer patch, and tests. Six repair systems and three LLMs were tested in 18 configurations. Many failed patches still left sensitive data reachable.
Unconfirmed: The claim that SIExBench is the first such benchmark is the authors' assessment.
- Forty-point-four percent of instances were unresolved by every configuration.
- Sixty-two-point-one percent of unsuccessful attempts still exposed sensitive information.
- Each instance includes exposure tests, regression tests, and a defect characterization.
- The authors say effective repair must close every exposure path.
From the original article
1 min read at arxiv.orgModern software systems routinely process credentials, personal records, and other sensitive information. A sensitive information exposure (SIEx) defect makes such information accessible to an unauthorized actor and can cause serious and often irreversible disclosure.
By Mingming Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.