ThinkBit preserves reasoning in scalar two-bit language models
Original title: ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning
On Qwen3-1.7B, a 0.52 GB recalibrated checkpoint scores 46.7 on MATH-500 versus 30.0 for EfficientQAT.
ThinkBit stores every weight as an integer codeword times a group-shared bfloat16 scale and assigns ternary, 2-bit, or 3-bit tiers by layer. It searches codewords, recalibrates frozen-codeword scales against a teacher, then runs short quantization-aware training. Authors report the QAT checkpoint at 73.0 on MATH-500 for Qwen3-1.7B, and a 3.74 GB model up to 14B at 94.8 versus 97.5 for a 29.5 GB bfloat16 model.
Unconfirmed: MATH-500 scores are reported by the authors, not independently verified here.
- A paired reasoning test chooses between the recalibration and QAT checkpoints.
- The 0.52 GB model is smaller than published 2-bit GGUFs of Qwen3-1.7B.
- Authors say the framework scales up to 14B parameters.
- QAT lets the model reassign codewords to match the base distribution.
From the original article
1 min read at huggingface.co
Reasoning language models generate thousands of tokens per answer, and generation speed is limited by memory bandwidth, so storing weights in two bits instead of bfloat16 can potentially make inference 8times cheaper. Two-bit methods are usually judged by perplexity and multiple-choice question-answering accuracy, both of which say little about whether a model can still reason through more difficult problems.
By Rhea Sanjay Sukthanker, Pashmina Cameron · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.