ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF — new model trending #3 on Hugging Face
ISTA-DASLab releases GSQ-RCO non-uniform GGUF quantizations of Qwen3.8-27B down to 2.5 bpw, with task-lossless IQ3_S matching BF16 benchmark scores.
ISTA-DASLab released GGUF quantizations of Qwen3.8-27B produced with GSQ (Gumbel-Softmax Quantization) and RCO (Riemannian Constrained Optimization), non-uniform methods that allocate per-tensor precision via gradient-based search under a total size budget. Four checkpoints range from 2.50 bpw (8.4 GB) to 3.50 bpw (11.8 GB), plus a BF16 vision projector (mmproj) enabling multimodal use. The recommended IQ3_S build is task-lossless, matching the BF16 base exactly on AIME25 (100.00) and LiveCodeBench v6 (85.71) at roughly one fifth of the BF16 size. Optional -mtp variants add a Multi-Token Prediction head for speculative decoding in llama.cpp.
- Four GGUF sizes from 2.50 to 3.50 bpw ship with a BF16 vision projector for multimodal use.
- GSQ handles low-bit scalar quantization; RCO assigns per-tensor quantization types under a size budget.
- IQ3_S is task-lossless, matching BF16 on AIME25 (100.00) and LiveCodeBench v6 (85.71).
- Optional -mtp builds add a Multi-Token Prediction head for speculative decoding.
- Files run unmodified in llama.cpp, Ollama, and LM Studio.
- Vendors
- ISTA-DASLabUnsloth
- Organizations
- Institute of Science and Technology Austria
- Countries
- Austria
- AI models
- Qwen3.8-27B
From the original article
6 min read at huggingface.coimage-text-to-text · 479,597 downloads · 619 likes · license apache-2. 0
By ISTA-DASLab · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.