ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF — new model trending #30 on Hugging Face
ISTA-DASLab releases GSQ-RCO non-uniform GGUF quantizations of Qwen3.8-Flash-Next, a 512-expert multimodal MoE, at 2.4-3.0 bits per weight.
The repository provides GGUF quantizations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 512 routed experts across 48 layers (10 active per token) plus a BF16 vision projector for multimodal use. Quantization combines GSQ (Gumbel-Softmax scalar quantization) and RCO (Riemannian constrained optimization), developed at IST Austria's DASLab, assigning quantization types per tensor via gradient-based search under a size budget. Three variants ship at 2.40-3.00 average bpw (66.4-75.8 GB total), and the recommended IQ3_XXS build matches the base model on AIME25 (100.00) while staying within 0.51 points on GPQA-Diamond and 1.14 on LiveCodeBench v6 at roughly one fifth of BF16 size. Files are standard GGUF running unmodified in llama.cpp, Ollama, and LM Studio under an Apache-2.0 license.