PIPETTE purple-teams prompt injection to harden LLM agents
Original title: PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents
On AgentDojo and ASB, defended agents reportedly near zero static attack success while keeping benign task utility.
PIPETTE is a prompt-injection purple-teaming framework for LLM agents that handle untrusted data and face indirect prompt injection. The authors say RL-Hammer and PISmith collapse onto similar attacks, so high red-team success does not improve later adversarial training. PIPETTE uses a dictionary-learning reward for broader attack coverage and an on-policy defense pipeline. On AgentDojo and ASB, defended agents reportedly resist static and strong adaptive attacks while keeping utility near the undefended baseline.
Unconfirmed: Near-zero attack success and robustness against adaptive attacks are the authors' evaluation results.
- RL-Hammer and PISmith mode collapse limited later robustness gains.
- The reward encourages broader coverage of attack strategies.
- Benign task utility stayed close to the undefended baseline.
From the original article
1 min read at huggingface.coLarge language model (LLM) agents routinely interact with untrusted external data, exposing them to indirect prompt injection (IPI) attacks that hijack agents into executing attacker-chosen tasks. To uncover these vulnerabilities at scale, automated red-teaming methods often leverage reinforcement learning (RL) to generate attacks that achieve high attack success rates (ASR).
By Lipeng He, Yihan Wang, Jiawen Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.