MACaQ quantizes weights and activations to MXFP4 layer by layer
Original title: MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations
On Qwen3-32B in vLLM it adds 2.3% overhead, and a 122B model was quantized in 64 minutes using 32.5GB GPU RAM.
MACaQ is an MXFP4 W4A4 post-training quantization method that treats every linear layer independently. It applies a per-channel scale and a block-Hadamard transform to activations online and absorbs inverse transforms into weights, so only one layer needs to be in memory. Flip-Refinement greedily adjusts individual FP4 codes. The authors report lower perplexity than DuQuant++ and MR-GPTQ, a match to their best Llama-3-8B perplexity in 11 times less quantization time, and use on an image diffusion model without architecture-specific changes.
- Activation transforms stay online, avoiding cross-layer dependencies.
- End-to-end overhead is 2.3% over pure MXFP4 on Qwen3-32B in vLLM.
- A 122B model was quantized in 64 minutes with 32.5GB GPU RAM.
- It was also applied to an image diffusion model.
- Products
- vLLM
- AI models
- Qwen3-32BLlama-3-8B
From the original article
1 min read at huggingface.co
Weight-activation quantization can reduce the memory and compute cost of neural-network inference, but recent methods often require expensive optimization and architecture-specific propagation of basis transforms across layers. We introduce MACaQ, an MXFP4 W4A4 post-training quantization method that treats every linear layer independently.
By Lorenz K. Muller, Philippe Bich, Chiara Boretti · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.