GitHub says extra instructions can limit AI security audits
Original title: Too many instructions can hold back AI security audits
A GitHub researcher says humans review every AI audit report and the team claims a 95% fix rate.
In a Help Net Security interview, GitHub security researcher Peter Stöckli describes how AI agents audit software, starting with authentication and authorization bugs. Taskflows split audits into narrow steps and begin with threat modeling of components, entry points and user actions. Prompts are tested per model, and unused instructions are dropped because overprompting can limit the model. A researcher validates every report, including a proof of concept, before it reaches maintainers.
Unconfirmed: The 95% fix rate is GitHub's own belief, not an independently verified figure.
- The first audit prompts focused on authentication and authorization flaws such as missing access checks.
- Threat-modeling output is rarely discarded, though researchers sometimes add data from other sources.
- Reports are sent from GitHub user accounts so maintainers have a point of contact.
- Lower-severity findings are filtered so maintainers are not sent weak reports.
- This year GitHub added rate limits, structured forms and API changes for report triage.
- Products
- GitHub Advisory Database
- Organizations
- GitHub
From the original article
3 min read at helpnetsecurity.com
In this Help Net Security interview, Peter Stöckli, Security Researcher at GitHub, explains how the team uses AI agents to audit software for vulnerabilities. The discussion covers why the work started with authentication and authorization bugs, how prompts are tested and trimmed for each model, and what the agent produces during threat modeling.
By Mirko Zorz · helpnetsecurity.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.