Ask-to-Distill learns to write questions for on-policy distillation
Original title: Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
A frozen student–teacher pair scores synthetic questions without reference answers, then those prompts feed standard on-policy distillation.
The paper introduces Ask-to-Distill, a framework that treats synthetic questions as distribution elicitors for on-policy distillation. A student-initialized synthesizer is trained with a distributional surrogate computed from a frozen student–teacher pair, using disagreement, teacher probability on the student’s support, and length and diversity controls, without reference answers or correctness rewards. Filtered, deduplicated prompts form a prompt-only bank. On math, code, and agentic tasks, the authors report better average performance and teacher-gap recovery than unoptimized student prompts and KL-filtered real questions, matching or beating real-bank distillation in the settings they evaluated. Code, data, and models are public.
Unconfirmed: Performance gains are the authors’ reported results and are not independently confirmed here.
- Question value is scored without reference answers or per-candidate student updates.
- Optimized prompts are filtered and deduplicated into a prompt-only bank.
- Authors report denser supervision from shorter student responses.
- Evaluated domains are mathematical reasoning, code generation, and agentic tasks.
- Code, data, and models are at github.com/GasolSun36/Ask_to_Distill.
From the original article
1 min read at huggingface.co
On-policy distillation (OPD) can succeed with surprisingly few training questions, motivating a view of questions as carriers of distributional knowledge transfer. How can we assess whether a newly generated question is valuable for distillation, and learn to generate better questions---without access to its correct answer?
By Jiashuo Sun, Hao Zhang, Xueqiang Xu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.