QDA uses agents to design closed-loop LLM quantization recipes
Original title: QDA: Closed-Loop LLM Quantization Design with Agents
It beats the strongest public recipe by 0.9 to 2.2 points and recovers 98.3% of FP8 accuracy on GLM-5.3-Flash.
QDA is a system in which an LLM agent designs post-training quantization recipes for each model and precision. It scores candidates in minutes by output KL divergence on unseen reasoning traces, then checks promising recipes with long generations. Across four reasoning models it improves average accuracy over the strongest public recipe at W4A16, W4A8 and W3A16, using 64 GPU-hours per search. On 320B GLM-5.3-Flash at NVFP4 W4A4 it recovers 98.3% of FP8 accuracy versus 96.9% for the best vendor checkpoint.
Unconfirmed: The planned release of the library, evaluator and harness is stated but not confirmed as available.
- Promising recipes are checked with long generations to catch repetition loops.
- Average gains are 0.9, 1.4 and 2.2 points at W4A16, W4A8 and W3A16.
- Each search uses 64 GPU-hours, and checkpoints add no inference overhead.
- Authors say they will release the library, evaluator and harness.
- Products
- QDA
- AI models
- GLM-5.3-Flash
From the original article
1 min read at huggingface.coPost-training quantization is the standard way to serve large language models at low cost. However, the best quantization recipe differs across models and precisions. As a result, experts must redesign it for every new model, which is time-consuming and often sub-optimal.
By Xingyang Li, Tianlang Zhao, Song Han · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.