Generalization behavior of OPTQ and the role of regularization
New bounds show OPTQ generalization depends on regularization lambda, and a revised lambda choice beats prior recommendations.
The paper studies generalization of OPTQ, which quantizes weights to minimize squared error on a calibration set, and a stochastic OPTQ variant. One result relates expected test squared error to calibration error when samples are drawn i.i.d. from the test distribution. Another bounds stochastic OPTQ for sufficiently regular distributions regardless of the calibration data. Regularization lambda is central to both bounds, and a newly recommended lambda outperforms prior choices in experiments.
- Expected test error tied to i.i.d. calibration-set error
- Stochastic OPTQ bounded for nice distributions independent of calibration
- Regularization lambda is central to both generalization bounds
- Recommended lambda outperforms earlier choices in experiments
From the original article
1 min read at arxiv.orgLarge neural networks can be compressed by rounding or "quantizing" their weights to numbers that admit representations with fewer bits. One algorithm for quantization, OPTQ, progressively quantizes the weights of a neural network so that the squared quantization error on a specified calibration dataset is as small as possible.
By Erin George · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.