On-Policy Distillation Lacks Generalization Beyond Supervised Data
Original title: On the Generalization Limitation of On-Policy Distillation for Post-RL Consolidation
MOPD does not fully transfer teachers' out-of-distribution capabilities from multiple teachers.
Multi-teacher on-policy distillation extends on-policy distillation to consolidate capabilities from multiple teachers. Evaluations across reasoning tasks reveal gaps in out-of-distribution performance compared to the base model. Geometry-based patching and base-preserving replay are proposed to address these gaps and reduce out-of-distribution forgetting.
- AI models
- Qwen3
From the original article
1 min read at huggingface.co
Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) to consolidate capabilities from multiple teachers. Although prior work reports near-perfect in-distribution (IND) distillation performance, we find that this can mask incomplete capability consolidation beyond the supervised distribution.
By Yichen Wang, Yufa Zhou, Siqi Zhu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.