PIPETTE purple-teaming aims to harden LLM agents against prompt injection
November 20: Help Net Security adds university affiliations and Qwen3-4B results showing PISmith’s larger attack set produced a weaker defense than RL-Hammer, while PIPETTE stock injections stayed under 0.5%.
Researchers say PIPETTE-trained agents resist static prompt injections on AgentDojo while keeping benign task utility near the baseline.
Researchers introduce PIPETTE, a prompt-injection purple-teaming framework for LLM agents that process untrusted data and face indirect prompt injection. They say RL-Hammer and PISmith collapse onto similar attacks, so a larger haul of successful injections does not by itself improve later adversarial training. PIPETTE uses a dictionary-learning reward for broader strategy coverage and an on-policy pipeline that checks actions still serve the user’s request. On AgentDojo and ASB, the authors report near-zero success on static attacks, resistance to strong adaptive attacks, and benign utility close to the undefended baseline. Help Net Security reports University of Waterloo, Zhejiang University and KTH tests in which a Qwen3-4B model retrained on PISmith’s larger set was easier to hijack than one trained with RL-Hammer.
- Help Net Security names Waterloo, Zhejiang University and KTH.
- PISmith broke 68.75% of 560 tasks and returned 7,132 attacks.
- RL-Hammer broke 33.39% of tasks and returned 4,282 attacks.
- PISmith-retrained Qwen3-4B was hijacked on 47.5% of unseen stress tests, versus 27.5%.
- Stock injections succeeded on under 0.5% of AgentDojo attempts against PIPETTE agents.
- Hardened Qwen3-4B finished 64.95% of clean tasks, versus 61.86% originally.
What each outlet adds
beyond the shared facts- Help Net SecurityAffiliations plus Qwen3-4B attack volumes, hijack rates and clean-task utility.
- arXiv cs.CRDictionary-learning reward, on-policy training, and AgentDojo and ASB results.
- Hugging Face daily papersMode collapse in RL-Hammer and PISmith limited later robustness gains.
Coverage
oldest first · one row per article · UTCPIPETTE purple-teams prompt injection to harden LLM agents
On AgentDojo and ASB, defended agents reportedly near zero static attack success while keeping benign task utility.
AutoPI red-teams LLM agents for prompt-injection technique coverage
Against Qwen3.6-35B it reports a successful injection in all six PIAtlas classes on AgentDojo.
PIPETTE turns prompt-injection red teaming into LLM agent defenses
Authors say RL-Hammer and PISmith collapse onto similar attacks that do not improve later training.
Larger prompt-injection red-team piles can weaken agent defenses
A Qwen3-4B retrained on PISmith’s larger haul was hijacked on 47.5% of unseen stress-test tasks, versus 27.5% for RL-Hammer.