AdvGuard detects multi-turn system prompt leakage with graph classification
Original title: Detecting and Mitigating Query-Based System Prompt Leakage Attacks
Authors report up to 36% higher recall than the strongest baseline, with a false-positive rate of at most 3.4%.
Researchers present AdvGuard, a model-agnostic framework that detects and mitigates query-based extraction of proprietary LLM system prompts. It builds a bounded semantic interaction graph across queries, filters candidate components, and classifies line graphs with a graph convolutional network, plus a lightweight prompt-level detector for short histories. In author evaluations, multi-turn workloads reach 97.23%–99.1% F1 with 100% recall, and integration into PLeak and LeakAgent loops sharply limits exact-match and substring leakage.
Unconfirmed: Detection and leakage figures are author-reported and not independently verified here.
- Leave-one-attack-family-out tests report 99.98% mean recall and 99.99% mean F1.
- Those tests report no observed false positives.
- Exact-match leakage is limited to 0.00–0.01 across evaluated victim models.
- Substring-match leakage is limited to 0.01–0.09 in the same loops.
- Products
- AdvGuard
From the original article
1 min read at arxiv.orgLarge Language Models (LLMs) rely on proprietary system prompts to enforce task directives, safety policies, and access controls. These prompts are vulnerable to query-based leakage attacks, where black-box adversaries extract confidential instructions through adaptive, multi-turn interactions. Existing defenses often assess queries independently, missing malicious behavior that emerges across interactions.
By Tushar Nayan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.