Skip to content
NewA calmer, faster ZeroHour

QDA uses agents to design closed-loop LLM quantization recipes

Original title: QDA: Closed-Loop LLM Quantization Design with Agents

Hugging Face daily papersby Xingyang Li, Tianlang Zhao, Song Han ()1 min read1
AI summary · x-ai/grok-4.7Summaries by Vani AI

It beats the strongest public recipe by 0.9 to 2.2 points and recovers 98.3% of FP8 accuracy on GLM-5.3-Flash.

QDA is a system in which an LLM agent designs post-training quantization recipes for each model and precision. It scores candidates in minutes by output KL divergence on unseen reasoning traces, then checks promising recipes with long generations. Across four reasoning models it improves average accuracy over the strongest public recipe at W4A16, W4A8 and W3A16, using 64 GPU-hours per search. On 320B GLM-5.3-Flash at NVFP4 W4A4 it recovers 98.3% of FP8 accuracy versus 96.9% for the best vendor checkpoint.

Unconfirmed: The planned release of the library, evaluator and harness is stated but not confirmed as available.

Key facts
  • Promising recipes are checked with long generations to catch repetition loops.
  • Average gains are 0.9, 1.4 and 2.2 points at W4A16, W4A8 and W3A16.
  • Each search uses 64 GPU-hours, and checkpoints add no inference overhead.
  • Authors say they will release the library, evaluator and harness.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at huggingface.co

Post-training quantization is the standard way to serve large language models at low cost. However, the best quantization recipe differs across models and precisions. As a result, experts must redesign it for every new model, which is time-consuming and often sub-optimal.

Hugging Face daily papers

By Xingyang Li, Tianlang Zhao, Song Han · huggingface.co

Read the full article at Hugging Face daily papers

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.