CIPL: A Channel-Aware Framework for Recoverable Privacy Leakage in LLM Agents
CIPL evaluates how much sensitive data in LLM-agent pipelines an external attacker can actually recover.
CIPL (Channel Inversion for Privacy Leakage) is a black-box framework that traces sensitive data through selection, assembly, execution, observation, and extraction to measure what an external observer can recover from LLM agents. Experiments cover memory-based, retrieval-mediated, and tool-mediated targets plus a BrowserUse live-agent case study. Memory targets were nearly saturated, retrieval leakage was often partial, and tool and live-agent leakage depended on observation surface, prompt alignment, retrieval depth, and provider behavior. A semantic audit found attacker-useful disclosures that exact-match scoring missed.
- CIPL measures externally recoverable leakage rather than internal storage exposure.
- Memory targets were near-saturated; retrieval leakage was often partial.
- Tool-mediated and BrowserUse leakage varied with observation surface and provider.
- Semantic audit caught useful disclosures missed by exact matching.
Full article170 words · extracted from arxiv.org · click to collapse
Privacy leakage in LLM agents is commonly evaluated within individual components such as memory, retrieval, or tool-use pipelines, which makes it difficult to distinguish internal exposure from information that an external observer can actually recover. We present CIPL (Channel Inversion for Privacy Leakage), a channel-aware evaluation framework for black-box privacy leakage in LLM agents. CIPL represents a target through sensitive source, selection, assembly, execution, observation, and extraction stages and evaluates the transition from selected sensitive units to attacker-recoverable output under a shared protocol. Experiments across memory-based, retrieval-mediated, and tool-mediated targets, together with a BrowserUse live-agent case study, show that storage labels alone do not determine recoverability. Memory targets form a near-saturated reference case, retrieval-mediated leakage is frequently partial, and tool-mediated and live-agent leakage varies strongly with observation surface, prompt-to-channel alignment, retrieval depth, and provider behavior. A stratified semantic audit further identifies attacker-useful disclosures that canonical exact matching misses. CIPL therefore provides a common framework for comparing how internal sensitive dependence is realized as externally recoverable leakage across heterogeneous agent pipelines.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.21686