Skip to content
NewA calmer, faster ZeroHour
StoryAI researchinfo

Entropy gate nearly matches 8-bit GSM8K on Qwen3-0.6B

2 sources · 3 articlesFirst reported Updated ()1
What's new

First merged summary.

Merged summary · x-ai/grok-4.7 · rewritten as coverage arrivesSummaries by Vani AI

A simulated entropy gate on Qwen3-0.6B scores 62.7% on 150 GSM8K items, close to always using 8-bit weights.

A paper stores simulated 8-bit weights for Qwen3-0.6B, decodes with the top 4 bits, and repeats a step at 8 bits when entropy or the top-two margin looks uncertain. Cost is weight bytes read per token. On 36,724 replayed steps, low-pass entropy ranks argmax flips with AUC 0.93; a 15% budget catches 78% of flips and the margin gate 94%. On 150 live GSM8K problems, all-high scores 64.0% and all-low 55.3%. A 16% entropy gate scores 62.7% while reading 0.87 of all-high bytes; its paired gap versus all-high is -1.3 points (p=0.080) and is not statistically separated. A live random control, a second model, and live KL were not run.

Key facts
  • Quantization is simulated on Qwen3-0.6B; cost is weight bytes read per token.
  • A 16% entropy gate reads 0.87 of the bytes used by always-8-bit decoding.
  • Versus all-low, the entropy gate is +7.3 points, with a CI from 0.0 to +14.7.
  • McNemar p-value for all-high versus all-low is 0.053.
  • At a 15% budget, the margin gate catches 94% of argmax flips.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

What each outlet adds

beyond the shared facts
  • Hugging Face daily papersReports the +7.3-point gap versus all-low and its confidence interval.
  • arXivAdds McNemar p=0.053 for all-high versus all-low.

Coverage

oldest first · one row per article · UTC
  1. · 6d ago Hugging Face daily papersFirst report

    Paper escalates Qwen3-0.6B from 4-bit to 8-bit on uncertain steps

    An entropy gate at a 16% escalation rate scores 62.7% on GSM8K versus 64.0% for always using 8-bit weights.

  2. · 6d ago Hugging Face daily papers

    6-bit Qwen3 weights plus order averaging beat extra bits

    Four cyclic answer orders recover 12.6 points at 4 bits on Qwen3-0.6B, versus 5.3 points at fp16.

  3. · 5d ago arXiv cs.AI / cs.LG / cs.CL

    Qwen3-0.6B mixes 4-bit decode with selective 8-bit steps

    An entropy gate at a 16% escalation rate scores 62.7% on 150 GSM8K problems, using 0.87 of all-high bytes.