4-bit rounding error, not bit width, shifts chat-model refusal
Original title: Is 4-Bit Quantization More Than Noise to a Chat Model's Safety Behaviour? Attributing Behavioural Shifts with Matched Perturbations
Qwen2.5-3B continues harmful replies more often, while Llama-3.2-3B complies 10.3 points less on held-out jailbreaks.
A pre-registered study of three 3–4B chat models tests whether 4-bit refusal changes come from a specific rounding error or from any equally large weight perturbation. With a compliant reply prefilled, Qwen2.5-3B continues into harmful content more often; on held-out jailbreaks, Llama-3.2-3B complies 10.3 points less often. Shuffled and Gaussian errors average near zero, while scaling the real error changes behaviour near-linearly. llama.cpp Q4_K_M, at the same bits per weight, moves neither model.
- At half the real error, Llama's likelihood is almost unchanged but compliance falls 5.8 points.
- An 8-bit null and paired tests with blind human labels were used.
- The authors say each 4-bit build must be compared with its original.
- Products
- llama.cpp
- AI models
- Qwen2.5-3BLlama-3.2-3B
From the original article
1 min read at arxiv.orgChat models on phones and laptops are usually 4-bit copies of the model that was evaluated, and audits of such copies disagree about what quantization does to refusal. A 4-bit build is one of many weight perturbations of the same size, so a change in its behaviour could come from its specific rounding error or from any perturbation that large.
By Ege Kaya · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.