ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF — new model trending #30 on Hugging Face
ISTA-DASLab releases GSQ-RCO non-uniform GGUF quantizations of Qwen3.8-Flash-Next, a 512-expert multimodal MoE, at 2.4-3.0 bits per weight.
The repository provides GGUF quantizations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 512 routed experts across 48 layers (10 active per token) plus a BF16 vision projector for multimodal use. Quantization combines GSQ (Gumbel-Softmax scalar quantization) and RCO (Riemannian constrained optimization), developed at IST Austria's DASLab, assigning quantization types per tensor via gradient-based search under a size budget. Three variants ship at 2.40-3.00 average bpw (66.4-75.8 GB total), and the recommended IQ3_XXS build matches the base model on AIME25 (100.00) while staying within 0.51 points on GPQA-Diamond and 1.14 on LiveCodeBench v6 at roughly one fifth of BF16 size. Files are standard GGUF running unmodified in llama.cpp, Ollama, and LM Studio under an Apache-2.0 license.
- IQ3_XXS variant matches the base model on AIME25 (100.00) at about one fifth of BF16 size.
- GSQ and RCO jointly select per-tensor quantization types via gradient-based search under a size budget.
- Q2_0 build delivers 3.4x prompt throughput and 1.9x lower latency than IQ2_XS with minor quality loss.
- Includes a BF16 vision projector (mmproj) for multimodal inference; Apache-2.0 licensed.
- Base model is a 512-expert sparse MoE across 48 layers with 10 experts active per token.
- Organizations
- ISTA-DASLab
- Countries
- Austria
- AI models
- Qwen3.8-Flash-Next
From the original article
9 min read at huggingface.coimage-text-to-text · 31,099 downloads · 148 likes · license apache-2. 0
By ISTA-DASLab · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.