AutoPI red-teams LLM agents for prompt-injection technique coverage
Original title: AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM Agents
Against Qwen3.6-35B it reports a successful injection in all six PIAtlas classes on AgentDojo.
Researchers present AutoPI, a black-box, training-free red teamer that uses the PIAtlas taxonomy to seek many distinct prompt-injection techniques rather than maximizing attack success alone. For each scenario an attacker model writes an injection for every sub-technique and revises failures using an LLM judge. On AgentDojo and IPI Arena, the authors report broader coverage than recent adaptive red teamers, including higher success and class coverage against Claude Haiku 5.5. They release code and generated attacks.
Unconfirmed: ASR and coverage gains versus other red teamers are author-reported.
- AutoPI replaces fixed benchmark templates and needs no training.
- Coverage counts taxonomy classes, techniques, and sub-techniques with a success.
- More than 90 percent of techniques succeeded against Qwen3.6-35B on AgentDojo.
- Claude Haiku 5.5 was the most robust target evaluated.
- AI models
- Qwen3.6-35BClaude Haiku 5.5
From the original article
1 min read at arxiv.orgPrompt injection is the dominant unsolved security risk for LLM agents, and the community now agrees that robustness must be measured against adaptive attackers rather than fixed benchmark injections. Many adaptive red teamers exist, and most are built to maximize attack success rate (ASR). Yet ASR counts how often a target breaks, not in how many different ways.
By Stavros Damianakis · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.