SoloQ: Calibration-Free Quantization for Diffusion Language Models
SoloQ quantizes diffusion language models to 4-bit without calibration, cutting memory up to 2.61x with NVFP4.
SoloQ is a calibration-free post-training quantization method for diffusion language models whose activations shift across masking states and denoising steps. It rotates weights and activations into a normalized basis with a predictable distribution, using structured K-RPBH rotation and a light rescaling correction, and also quantizes block-diffusion KV caches at commit time. Across LLaDA, Dream, Fast-dLLM v2, and Nemotron-Labs-Diffusion, 4-bit SoloQ keeps accuracy and beats calibration-based baselines on knowledge and reasoning benchmarks. With NVFP4 it reduces peak memory by up to 2.61x and speeds end-to-end inference by up to 2.24x.