RAG-PIBench: A Leakage-Aware Benchmark for Prompt-Injection Detection in Trustworthy RAG Systems
RAG-PIBench offers 4,876 leakage-aware examples for detecting prompt injection in RAG systems.
The paper introduces RAG-PIBench, a leakage-aware benchmark for detecting prompt injection embedded in retrieved content for RAG systems. It contains 4,876 contextual examples split into frozen train, validation, and protected-test sets. DistilBERT records the best protected-test scores, with F1 0.896 and PR-AUC 0.968, while TF-IDF SVM and logistic regression stay competitive. The authors argue that leakage-aware design and strong sparse baselines improve reliable detection.
- Benchmark contains 4,876 examples with frozen train, validation, and protected-test splits.
- DistilBERT leads the protected test with F1 0.896 and PR-AUC 0.968.
- TF-IDF SVM and logistic regression remain competitive sparse baselines.
- A leakage-aware pipeline is used to limit train-test contamination.
From the original article
1 min read at arxiv.orgRetrieval-Augmented Generation (RAG) systems are vulnerable to prompt-injection attacks embedded in retrieved content. We introduce RAG-PIBench, a benchmark for RAG-style prompt-injection detection containing 4,876 contextual examples across frozen train, validation, and protected-test splits.
By Niveen O. Jaffal · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.