Defensive Sufficiency in a Stackelberg Model of AI Security
A Stackelberg model finds when red-team feedback and repairs suffice to defend finite AI attack surfaces.
The paper studies when automated testing, human red teaming, and incident response yield sufficient AI defenses and when that investment pays off. It shows a finite attack surface is defended with probability 1 if unresolved attacks remain discoverable, repairs are effective, and later updates preserve earlier fixes, and it derives completion-time bounds for growing surfaces and multiple discovery mechanisms. A defender-led Stackelberg game characterizes the least-cost mix of proactive discovery and reactive repair that deters attacker search. Numerical experiments show faster repair can shorten compromise duration without reducing compromise probability.
- Probability-1 defense needs persistent discovery, effective repairs, and preserved fixes
- Gives completion-time bounds for growing surfaces and multiple detectors
- Stackelberg analysis covers funding neither, one, or both capabilities
- Faster repair can shorten compromises without lowering their probability
From the original article
1 min read at arxiv.orgFeedback from automated testing, human red teaming, and incident response can strengthen an AI system's defenses when discovered failures lead to effective repairs. We study when this feedback process provides sufficient protection and when investing in it is economically worthwhile.
By Subhabrata Majumdar · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.