Skip to content
NewA calmer, faster ZeroHour

LastOPD: Taming Collapse in Latent On-Policy Distillation

Hugging Face daily papersby Jie Yang, Zhengyu Fang, Zelin Xu ()1 min read3 · 2 reads
AI summary · grok-4.7Summaries by Vani AI

LastOPD briefly aligns only the last-layer latent state when distilling Qwen3 into Qwen3-1.7B, beating token-only OPD on MATH-500.

Full latent on-policy distillation of Qwen3-4B and Qwen3-8B into Qwen3-1.7B-Base raised MATH-500 from 25 to 46 in 10 steps, then collapsed to 11 even as the alignment metric improved. The authors argue depth-paired layers play different roles, so continued alignment pulls the student toward teacher states it cannot use. LastOPD applies the latent signal only at the last-layer state during a 10-step crossfade into token-level OPD. It beats token-only OPD by 5.55 and 4.02 MATH-500 points with the 4B and 8B teachers and reaches that baseline's final score in about half the steps.

Key facts
  • Latent supervision lifted MATH-500 from 25 to 46, then collapsed it to 11.
  • Alignment kept improving while the most aligned student performed worst.
  • LastOPD applies latent loss only at the last layer for a 10-step crossfade.
  • MATH-500 rose 5.55 and 4.02 points over token-only OPD with 4B and 8B teachers.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at huggingface.co

On-policy distillation (OPD) corrects a student on the responses it writes, but its signal is the teacher's next-token distribution: it tells the student what the teacher says but misses how it thinks. Latent supervision promises the missing part by aligning the student's latent states to the teacher's. Recent methods such as OPRD bring this signal into on-policy distillation.

Hugging Face daily papers

By Jie Yang, Zhengyu Fang, Zelin Xu · huggingface.co

Read the full article at Hugging Face daily papers

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.