SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
SQS unifies weight pruning and low-bit quantization via Bayesian variational learning, compressing Llama3.2 and Qwen2.5 at higher rates with comparable accuracy.
SQS introduces a unified Bayesian variational framework performing simultaneous pruning and low-bit quantization, using a spike-and-slab prior for sparsity and Gaussian Mixture Models to model quantized weights. The authors derive an efficient approximation for the intractable objective and provide a consistency result for the variational approach. Experiments on ResNet, BERT-base, Llama3.2, and Qwen2.5 show higher compression rates than prior baselines with comparable performance drops.
28