Trail of Bits ships agent plugins for patches and regression checks
Original title: Your AI agent fixed the bug, but what else did it change?
A review of over 130 merged PRs found 21 regressions, none of which broke a security protection.
Trail of Bits published patching-bugs and regression-hunt, plugins that have agents reproduce bugs, write fixes, and check whether changes break previously working behavior. They were built through Patch the Planet with OpenAI, drawing on merged patches and Codex sessions, including work on Scapy. Maintainers can set security assumptions and review results. Across the reviewed set, regression-hunt produced 25% fewer false-positive findings than a baseline Codex agent asked to review a pull request for regressions.
Unconfirmed: The 25% false-positive reduction is from Trail of Bits’ preliminary evaluation.
- Review found 21 regressions: 16 functional, one performance, four build or test.
- Those 21 came from 58 candidates in one run, about 36% precision.
- Each claim needs a live proof of concept or the result is inconclusive.
- Plugins use fixed schemas and optional project threat models.
- Organizations
- Trail of BitsOpenAI
From the original article
8 min read at blog.trailofbits.com
Today we’re publishing patching-bugs and regression-hunt , two plugins that let engineers delegate patch development and regression checks to agents. Our patching-bugs plugin reproduces reported bugs, develops fixes, and tests them. The regression-hunt plugin checks whether any changes break previously working and supported behavior.
By blog.trailofbits.com via sarciszewski · blog.trailofbits.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.