EDIC scores each external instruction before an LLM complies
Original title: Identify, Classify, Act: Per-Instruction Controlled Compliance for LLMs over External Data
On Qwen3-4B, the paper reports up to 7.7% better harmful rejection and 12.5% higher Arena-Hard-Auto utility.
A paper argues that LLMs using external data need per-instruction credit attribution because one document or tool return can mix legitimate directives with prompt injections. It formalizes External-Data Instruction Classification and proposes EDIC, a decomposed reward that encourages beneficial compliance and penalizes harmful compliance. A scene-grounded pipeline injects mixed instructions with counterfactual filtering; a generative judge is initialized by supervised fine-tuning and refined with GRPO. Across two backbones and five benchmarks, the authors say EDIC beats the strongest prior defense on both safety and utility. Code is hosted on GitHub.
Unconfirmed: Benchmark gains are the authors' reported comparisons, not an independent evaluation.
- The judge attributes compliance at the instruction level for RL training.
- Reported gains are up to 7.7% harmful-rejection and 12.5% Arena-Hard-Auto utility.
- Results are stated for Qwen3-4B among two backbones.
- Training uses SFT initialization followed by GRPO.
- AI models
- Qwen3-4B
From the original article
1 min read at arxiv.orgLLMs over external data face a per-instruction tension: a single document or tool return often interleaves legitimate directives with adversarial prompt injections, and the model must follow the former while rejecting the latter. Existing defenses suppress or trust all external instructions, collapsing the safety-utility frontier and forcing a choice between over-refusal and over-compliance.
By Zesheng Shi · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.