Entropy gate nearly matches 8-bit GSM8K on Qwen3-0.6B
First merged summary.
A simulated entropy gate on Qwen3-0.6B scores 62.7% on 150 GSM8K items, close to always using 8-bit weights.
A paper stores simulated 8-bit weights for Qwen3-0.6B, decodes with the top 4 bits, and repeats a step at 8 bits when entropy or the top-two margin looks uncertain. Cost is weight bytes read per token. On 36,724 replayed steps, low-pass entropy ranks argmax flips with AUC 0.93; a 15% budget catches 78% of flips and the margin gate 94%. On 150 live GSM8K problems, all-high scores 64.0% and all-low 55.3%. A 16% entropy gate scores 62.7% while reading 0.87 of all-high bytes; its paired gap versus all-high is -1.3 points (p=0.080) and is not statistically separated. A live random control, a second model, and live KL were not run.
- Quantization is simulated on Qwen3-0.6B; cost is weight bytes read per token.
- A 16% entropy gate reads 0.87 of the bytes used by always-8-bit decoding.
- Versus all-low, the entropy gate is +7.3 points, with a CI from 0.0 to +14.7.
- McNemar p-value for all-high versus all-low is 0.053.
- At a 15% budget, the margin gate catches 94% of argmax flips.
What each outlet adds
beyond the shared facts- Hugging Face daily papersReports the +7.3-point gap versus all-low and its confidence interval.
- arXivAdds McNemar p=0.053 for all-high versus all-low.
- AI models
- Qwen3-0.6BQwen3-1.7B
Coverage
oldest first · one row per article · UTCPaper escalates Qwen3-0.6B from 4-bit to 8-bit on uncertain steps
An entropy gate at a 16% escalation rate scores 62.7% on GSM8K versus 64.0% for always using 8-bit weights.
6-bit Qwen3 weights plus order averaging beat extra bits
Four cyclic answer orders recover 12.6 points at 4 bits on Qwen3-0.6B, versus 5.3 points at fp16.
Qwen3-0.6B mixes 4-bit decode with selective 8-bit steps
An entropy gate at a 16% escalation rate scores 62.7% on 150 GSM8K problems, using 0.87 of all-high bytes.