Origin Is All You Need: Provenance-Aware Transformers for Structural Trust-Boundary Separation
Provenance-aware transformers use origin labels to structurally block indirect prompt injection.
The paper argues ordinary transformers cannot separate system instructions from retrieved or user content because one attention mechanism processes every token, enabling indirect prompt injection. Provenance-Aware Transformers assign each token a ring ID and add origin embeddings, a learnable origin attention bias, and an origin scale that survives normalization. A two-stage fine-tune teaches released pretrained models those origin semantics under ring constraints. The authors report robust in-distribution and out-of-distribution resistance to indirect prompt injection while utility stays comparable to the base model.
- Indirect prompt injection exploits attention that treats every source alike.
- Ring IDs, origin embeddings, attention bias, and scale encode provenance.
- Two-stage fine-tuning adapts released pretrained models to ring constraints.
- Reported injection resistance holds in and out of distribution with similar utility.
From the original article
1 min read at arxiv.orgIndirect prompt injection (IPI) remains a central safety and security challenge for large language model (LLM) systems because standard transformers lack architectural notion of source authority.
By Yuxuan Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.