LLMs and Contextual Integrity
Bruce Schneier highlights two papers: the CIMemories benchmark shows frontier LLMs leak memory attributes up to 69%, and an RL method reduces inappropriate disclosures.
Bruce Schneier discusses contextual integrity in LLMs, referencing the CIMemories benchmark, which uses synthetic profiles with 100+ attributes per user to test whether models with persistent memory disclose sensitive information appropriately. Evaluation showed frontier models exhibit up to 69% attribute-level violations, with GPT-5's violation rate rising from 0.1% to 9.6% across 40 tasks and reaching 25.1% with repeated prompting, showing unstable leakage behavior. A second paper introduces a reinforcement learning framework trained on a synthetic 700-example dataset that substantially reduces inappropriate disclosure while maintaining task performance, with improvements transferring to the human-annotated PrivacyLens benchmark.
- CIMemories benchmark tests information flow control from LLM persistent memory across 100+ attribute profiles
- Frontier models showed up to 69% attribute-level privacy violations, often trading task utility for fewer leaks
- GPT-5 violations grew from 0.1% to 9.6% over 40 tasks and hit 25.1% with 5 repeated identical prompts
- RL framework on 700 synthetic examples cut inappropriate disclosure and transferred to the PrivacyLens benchmark
- Products
- CIMemoriesPrivacyLens
- AI models
- GPT-5
From the original article
2 min read at schneier.comI have been thinking a lot about AI and integrity. Part of that is contextual integrity. I recently found two papers on the topic.
By Bruce Schneier · schneier.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.