Benchmarking Qwen3.8 27B quantizations: 4-bit holds up, 1-bit collapses
Quesma benchmarks Qwen3.8 27B quantizations: 4-bit Q4_K_M matches BF16 on key benchmarks while 1-bit collapses to random chance.
Quesma spent roughly $3,000 on Modal GPUs testing Unsloth GGUF quantizations of Qwen3.8 27B across GPQA Diamond, IFBench, and Terminal-Bench 2.1. The 17 GB Q4_K_M quantization matched the 55 GB BF16 model on Terminal-Bench 2.1 and showed little degradation down to 4-bit, while the 2-bit UD-Q2_K_XL dropped noticeably. At 1-bit, scores on GPQA Diamond fell to random-guess levels, with longer reasoning making results worse, and reasoning effort settings significantly affected outcomes.
- Q4_K_M (17 GB) matches BF16 on Terminal-Bench 2.1
- 2-bit UD-Q2_K_XL shows noticeable but usable degradation
- 1-bit models score near random chance on GPQA Diamond
- Higher reasoning effort amplified 1-bit failures
- Products
- llama.cpp
- AI models
- Qwen3.8 27B
From the original article
6 min read at quesma.com
How much GPU RAM do you actually need to run Qwen3. 8 27B without sacrificing quality? The full BF16 model weighs 55 GB, putting it beyond most consumer hardware.
By stared · quesma.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.