Skip to content
NewA calmer, faster ZeroHour

6-bit Qwen3 weights plus order averaging beat extra bits

Original title: Bits or permutations? Spending a fixed inference budget on weight precision or on option-order averaging in small local language models

Hugging Face daily papersby Mohammed Nayeem Khan, Logesh L, Aliyas Ahmad Misri ()1 min read1
Part of a story covered by 3 sourcesEntropy gate nearly matches 8-bit GSM8K on Qwen3-0.6BMerged summary & timeline
AI summary · x-ai/grok-4.7Summaries by Vani AI

Four cyclic answer orders recover 12.6 points at 4 bits on Qwen3-0.6B, versus 5.3 points at fp16.

The paper compares round-to-nearest weight quantization with averaging reordered multiple-choice options on Qwen3-0.6B and Qwen3-1.7B. Against an fp16 reference, 6-bit weights lose no accuracy and 8-bit weights change it only slightly, while 4-bit weights raise answer-slot skew. In a cost unit of weight bytes read times passes, extra orders beat extra bits only at 6 bits or more, so the authors recommend 6-bit weights and up to four one-pass orders. The study is simulated with the simplest quantizer on two small models.

Unconfirmed: The authors do not explain the 6-bit accuracy gain, and the noise control does not settle whether rounding adds slot skew.

Key facts
  • Benchmarks are ARC-Easy, ARC-Challenge, MMLU and HellaSwag, 300 items each.
  • Six-bit scores are +1.8 and +0.9 points versus fp16 on the two models.
  • At 4 bits, slot skew rises from 0.15 to 0.53 on the 0.6B model.
  • A Gaussian-noise control at the same weight SNR is inconclusive.
  • A title search found no earlier joint study of quantization and option order.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at huggingface.co

Local language models can save memory by storing weights in fewer bits, and they can reduce sensitivity to the order of answer options by averaging a few reorderings. We ask whether the first makes the second worse, and which is the cheaper way to spend a fixed budget. On Qwen3-0.

Hugging Face daily papers

By Mohammed Nayeem Khan, Logesh L, Aliyas Ahmad Misri · huggingface.co

Read the full article at Hugging Face daily papers

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.