What Does Privileged Information Add to On-Policy Self-Distillation?
Study of on-policy self-distillation on 5,319 AMPLE-Math problems finds reference-free distillation drives most gains, with modest extra benefit from privileged references.
The authors build AMPLE-Math, a suite of 5,319 mathematical problems with six reasoning views sharing the same answers, to isolate the value of privileged references in on-policy self-distillation. With a thinking-enabled teacher supervising direct-response rollouts, reference-free distillation accounts for most of Qwen3-1.7B's improvement in domain and on external benchmarks; extra reference benefit is modest, strongest for polished solutions, while complete traces add two percentage points for SmolLM3-3B at step 50. Gains reverse when short direct-response rollouts are replaced with long thinking-enabled rollouts. The findings suggest OPSD improves access to existing reasoning capabilities via cross-mode parameter sharing, with references valued for what they add to that transfer.
- Introduces AMPLE-Math: 5,319 math problems with six reasoning views sharing the same answers.
- Reference-free distillation accounts for most of Qwen3-1.7B's improvement in domain and external benchmarks.
- Privileged-reference benefits are modest in Qwen; complete traces add two points for SmolLM3-3B.
- Gains reverse when short direct-response rollouts are replaced with long thinking-enabled rollouts.
- Products
- AMPLE-Math
- AI models
- Qwen3-1.7BSmolLM3-3B
From the original article
1 min read at huggingface.co
On-policy self-distillation (OPSD) lets a language model learn from a frozen copy of itself that sees an answer or a worked solution. Giving the teacher this extra information seems to offer the student more to learn, but how much does it add beyond distillation itself?
By XiuYu Zhang, Wei Chow, Junfeng Fang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.