DuoOPD: Learning from Joint Teacher-Student Outcomes for Multi-Task On-Policy Distillation
DuoOPD uses joint teacher-student outcomes to improve multi-task on-policy distillation over standard OPD.
DuoOPD sets feedback direction from the student's outcome and uses the joint teacher-student outcome to decide how the teacher supports that direction. When only the teacher is correct, its verified answer becomes context for scoring the student's failed response; when only the student is correct, a task-shared weight reinforces the whole response. A single rule covers all four outcome combinations without task-specific settings. Across Qwen3 and Llama it beats five baselines, improving over on-policy distillation by 2.58 and 5.98 percentage points in mean macro accuracy, and also leads on mixtures spanning scientific calculation, instruction following, and code generation.
- Student outcome sets direction; joint outcome sets how the teacher helps.
- One rule covers all four teacher-student outcome combinations.
- Gains over OPD are 2.58 and 5.98 points on Qwen3 and Llama.
- Outcome direction alone stays near the gated baseline.
From the original article
1 min read at huggingface.co
On-policy distillation (OPD) trains a student on its own responses with token-level feedback from a stronger teacher, yet the teacher can fail on questions the student already answers correctly, and how often each model succeeds varies across tasks.
By Ao Yu, Weibo Gao, Heng Zhou · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.