SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
SQS unifies weight pruning and low-bit quantization via Bayesian variational learning, compressing Llama3.2 and Qwen2.5 at higher rates with comparable accuracy.
SQS introduces a unified Bayesian variational framework performing simultaneous pruning and low-bit quantization, using a spike-and-slab prior for sparsity and Gaussian Mixture Models to model quantized weights. The authors derive an efficient approximation for the intractable objective and provide a consistency result for the variational approach. Experiments on ResNet, BERT-base, Llama3.2, and Qwen2.5 show higher compression rates than prior baselines with comparable performance drops.
- Spike-and-slab prior induces sparsity; GMMs model low-bit quantized weights.
- Higher compression rates than existing baselines with comparable accuracy loss.
- Validated on ResNet, BERT-base, Llama3.2, and Qwen2.5 models.
- Includes theoretical consistency result for the variational approximation.
From the original article
1 min read at huggingface.co
Compressing large-scale neural networks is essential for deploying models on resource-constrained devices. Most existing methods adopt weight pruning or low-bit quantization individually, often resulting in suboptimal compression rates to preserve acceptable performance drops.
By Ziyi Wang, Nan Jiang, Guang Lin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.