One KV-cache bit flip turns correct LLM answers wrong
Original title: From Token Changes to Task Failures: Bit Flips in a Shared KV-Cache Block
On Qwen3-8B in vLLM, mantissa or sign flips made correct TriviaQA answers wrong about 0.59 to 0.60 of the time, versus about 0.02 on MBPP+.
The paper measures silent task failures from a single bit flip in a shared prefix-cache KV block used by every request with that system prompt. Software fault injection into vLLM serving Qwen3-8B flips the most significant mantissa bit or the sign bit. Harm concentrated on requests whose sampled answers are sometimes wrong, and the wrong answers looked like ordinary errors, so the authors say detection should check cached blocks rather than outputs.
- Tasks were code generation, short answers, multiple-choice reasoning and policy refusal.
- Mantissa-bit wrong-answer fractions were 0.023 on MBPP+ and 0.59 on held-out TriviaQA.
- Sign-bit fractions were 0.019 and 0.60 on those same tasks.
- Earlier work had shown many later responses change without looking degenerate.
Impact: Silent wrong model answers from one shared KV-block bit flip
| Product | Affected |
|---|---|
| vLLM serving Qwen3-8B | — |
- 1.Check cached KV blocks rather than monitoring output text for degeneration.
From the original article
1 min read at arxiv.orgIn LLM serving systems with prefix caching, each KV block of a shared system prompt is held in GPU memory as a single physical copy, which every request with that prompt reads. Our earlier work measured change and showed that one bit flip in such a block changes many later responses without making them visibly degenerate. This paper extends that work by measuring how often the changed responses are wrong.
By Yuji Yamamoto · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.