Skip to content
NewA calmer, faster ZeroHour

One KV-cache bit flip turns correct LLM answers wrong

Original title: From Token Changes to Task Failures: Bit Flips in a Shared KV-Cache Block

arXiv cs.CRby Yuji Yamamoto ()1 min read
AI summary · x-ai/grok-4.7Summaries by Vani AI

On Qwen3-8B in vLLM, mantissa or sign flips made correct TriviaQA answers wrong about 0.59 to 0.60 of the time, versus about 0.02 on MBPP+.

The paper measures silent task failures from a single bit flip in a shared prefix-cache KV block used by every request with that system prompt. Software fault injection into vLLM serving Qwen3-8B flips the most significant mantissa bit or the sign bit. Harm concentrated on requests whose sampled answers are sometimes wrong, and the wrong answers looked like ordinary errors, so the authors say detection should check cached blocks rather than outputs.

Key facts
  • Tasks were code generation, short answers, multiple-choice reasoning and policy refusal.
  • Mantissa-bit wrong-answer fractions were 0.023 on MBPP+ and 0.59 on held-out TriviaQA.
  • Sign-bit fractions were 0.019 and 0.60 on those same tasks.
  • Earlier work had shown many later responses change without looking degenerate.
Security assessment · AI, from the article

Impact: Silent wrong model answers from one shared KV-block bit flip

ProductAffected
vLLM serving Qwen3-8B—
What to do
  1. 1.Check cached KV blocks rather than monitoring output text for degeneration.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

In LLM serving systems with prefix caching, each KV block of a shared system prompt is held in GPU memory as a single physical copy, which every request with that prompt reads. Our earlier work measured change and showed that one bit flip in such a block changes many later responses without making them visibly degenerate. This paper extends that work by measuring how often the changed responses are wrong.

arXiv cs.CR

By Yuji Yamamoto · arxiv.org

Read the full article at arXiv cs.CR

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.