Mirage bounds LLM KV-cache timing leaks without blocking sharing
Original title: Mirage: Provable Defense Against LLM KV Cache Timing Side Channels with Topology-Aware Isolation
On a 500-sample corpus and three models, token-emission delays pushed leakage near random guessing with lower TTFT than baselines.
Shared LLM services that reuse KV cache and prefixes can leak prompt tokens through time-to-first-token differences between cache hits and misses. Mirage delays token emission at the gateway so an observer sees a cache-miss-like distribution while the GPU keeps serving other requests. A Privacy Odometer tracks prefix fan-out and triggers padding after a safety threshold. The authors say it also resists a strong adaptive adversary and applies to semantic caching.
Unconfirmed: Performance and leakage results are from the authors' evaluation, not an independent deployment.
- Existing defenses restrict cross-tenant cache sharing and add large overhead.
- Mirage pads TTFT to a calibrated cache-miss baseline after the threshold.
- The paper says the GPU is not stalled and requests are not recomputed.
- Evaluation used a 500-sample public corpus across three unnamed models.
From the original article
1 min read at arxiv.orgLarge Language Models (LLMs) are increasingly deployed as shared multi-tenant services, relying on advanced optimizations like KV cache and prefix sharing to improve throughput and reduce latency.
By Xiaobei Yan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.