CIPL: A Channel-Aware Framework for Recoverable Privacy Leakage in LLM Agents
CIPL evaluates how much sensitive data in LLM-agent pipelines an external attacker can actually recover.
CIPL (Channel Inversion for Privacy Leakage) is a black-box framework that traces sensitive data through selection, assembly, execution, observation, and extraction to measure what an external observer can recover from LLM agents. Experiments cover memory-based, retrieval-mediated, and tool-mediated targets plus a BrowserUse live-agent case study. Memory targets were nearly saturated, retrieval leakage was often partial, and tool and live-agent leakage depended on observation surface, prompt alignment, retrieval depth, and provider behavior. A semantic audit found attacker-useful disclosures that exact-match scoring missed.
- CIPL measures externally recoverable leakage rather than internal storage exposure.
- Memory targets were near-saturated; retrieval leakage was often partial.
- Tool-mediated and BrowserUse leakage varied with observation surface and provider.
- Semantic audit caught useful disclosures missed by exact matching.
From the original article
1 min read at arxiv.orgPrivacy leakage in LLM agents is commonly evaluated within individual components such as memory, retrieval, or tool-use pipelines, which makes it difficult to distinguish internal exposure from information that an external observer can actually recover. We present CIPL (Channel Inversion for Privacy Leakage), a channel-aware evaluation framework for black-box privacy leakage in LLM agents.
By Tao Huang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.