SoloQ: Calibration-Free Quantization for Diffusion Language Models
SoloQ quantizes diffusion language models to 4-bit without calibration, cutting memory up to 2.61x with NVFP4.
SoloQ is a calibration-free post-training quantization method for diffusion language models whose activations shift across masking states and denoising steps. It rotates weights and activations into a normalized basis with a predictable distribution, using structured K-RPBH rotation and a light rescaling correction, and also quantizes block-diffusion KV caches at commit time. Across LLaDA, Dream, Fast-dLLM v2, and Nemotron-Labs-Diffusion, 4-bit SoloQ keeps accuracy and beats calibration-based baselines on knowledge and reasoning benchmarks. With NVFP4 it reduces peak memory by up to 2.61x and speeds end-to-end inference by up to 2.24x.
- SoloQ quantizes weights and activations without calibration data.
- K-RPBH rotation plus rescaling supports matched codebooks and NVFP4.
- Tested on LLaDA, Dream, Fast-dLLM v2, and Nemotron-Labs-Diffusion.
- NVFP4 cuts peak memory up to 2.61x and speeds inference up to 2.24x.
Full article188 words · extracted from huggingface.co · click to collapse
Diffusion large language models dLLMs) have emerged as a promising alternative to autoregressive language models through bidirectional diffusion-based token generation. However, their growing model sizes and high inference costs make efficient deployment challenging: full-sequence denoising repeatedly invokes compute-intensive forward passes, while block-diffusion models additionally introduce a memory-intensive KV-cache. Low-bit weight-activation quantization is therefore attractive, yet existing dLLM post-training quantization methods rely on calibration data despite activation distributions shifting across masking states and denoising steps. We present SoloQ, a calibration-free quantization framework that maps weights and activations into a normalized rotated basis with a predictable marginal distribution, enabling data-independent quantization. SoloQ combines a structured K-RPBH rotation with a lightweight rescaling correction for calibration-free quantization. Its predictable post-rotation distribution supports both distribution-matched codebooks and hardware-native NVFP4. For block-diffusion models, SoloQ further applies commit-time KV-cache quantization to compress persistent states without perturbing the actively denoised block. Across full-sequence dLLMs (LLaDA and Dream) and block-diffusion dLLMs(Fast-dLLM v2 and Nemotron-Labs-Diffusion), SoloQ retains accuracy under 4-bit quantization and outperforms calibration-based baselines on knowledge- and reasoning-intensive benchmarks. With NVFP4, SoloQ reduces peak memory by up to 2.61X and accelerates end-to-end inference by up to 2.24X.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2610.07121