G^2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation
G2PTQ refreshes gradients and Hessians during LLM quantization, beating GPTQ-style baselines.
G^2PTQ is a post-training quantization framework that combines first-order gradients and second-order Hessian information under a globally supervised, block-wise objective. Gradient and Hessian estimates are refreshed before each Transformer block, and a trust-region scaling mechanism bounds the gradient step to stabilize exact first-order compensation. The authors also derive efficient block-wise Hessian approximation and exact gradient compensation. Experiments across model families and bit-widths show closer alignment with full-precision models than prior GPTQ-style baselines, with code released.
- G2PTQ combines refreshed first-order gradients with Hessian estimates under a global block-wise objective.
- A trust-region scale bounds the gradient step to avoid exploding weight updates.
- It refreshes estimates before each Transformer block, avoiding stale GPTQ-style Hessians.
- Results across model families and bit-widths beat state-of-the-art PTQ baselines; code is public.
From the original article
1 min read at huggingface.co
Post-training quantization (PTQ) is a practical approach to reducing the memory and computational footprint of large language models (LLMs) without retraining. GPTQ-based methods have become the de facto standard, yet they suffer from two complementary limitations.
By Ruikang Liu, Haoli Bai, Yuxuan Sun · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.