Benchmarking indirect prompt injection with task-tampering attacks
Original title: Beyond Instruction Override: Benchmarking Indirect Prompt Injection with Task-Tampering Attacks
New benchmark tests task-tampering in indirect prompt injection across 90 cases in three tool-use environments, showing higher success on decision rules and actions than objective attacks.
Indirect prompt-injection benchmarks test planted instructions redirecting agents to attacker-chosen tasks. The paper defines indirect prompt injection by authority: planted content influences a decision the source is not authorized to determine. It introduces a benchmark with 90 validated cases where user requests specify the source for targeted decisions. Attacks tamper with one of five targets in the agent's decision making without replacing the task. Without defenses, attacks on decision rules, actions, and state have higher success rates than objective attacks across five LLMs.
- Defines IPI by authority rather than instruction wording
- Benchmark spans 90 cases across three tool-use environments
- Attacks target objective, rules, actions, observations, or state
- Detection defense reduces objective attacks to 2.4% but leaves others higher
- AI models
- LLMs
From the original article
1 min read at arxiv.orgIndirect prompt-injection (IPI) benchmarks predominantly test planted instructions that redirect a tool-using agent toward an attacker-chosen task. Yet attacker-controlled content can instead tamper with the outcome of the task already in progress. A buyer's dispute evidence can cite a false refund policy to inflate a refund without changing the task.
By Rongting Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.