NTBench shows higher success for near-target prompt injection
Original title: On Task, Off Track: Benchmarking LLM and Agent Robustness to Near-Target Prompt Injection
On language tasks, incorrect-target success rises 9.3 to 35.8 points versus a fixed-target baseline.
NTBench builds prompt-injection cases from legitimate tasks so answers stay close to a reference outcome but become incorrect or harmful, including in agent tool calls. Seven LLMs are tested on ten language datasets and two agent benchmarks, with three attack executions per sample. Both tracks beat the fixed-target baseline for every model. Harmful targets may sit alongside otherwise correct task completion, and deviations already seen without injection are excluded.
Unconfirmed: The seven LLMs and the two agent benchmarks are not named in the abstract.
- Harmful-target gains on language tasks are 1.7 to 17.3 percentage points.
- Checks mix LLM judgments with human validation of intended deviations.
- The budget is three attack executions per sample.
- Fixed-target resistance did not imply resistance to near-target outcomes.
From the original article
1 min read at arxiv.orgPrompt injection benchmarks assess whether large language models (LLMs) and agents resist malicious instructions embedded in external content. However, a system could perform well by rejecting attacks that obviously abandon the user's request, while still accepting changes that retain most of a legitimate answer but make it incorrect or harmful.
By Yuyang Gong · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.