LastOPD: Taming Collapse in Latent On-Policy Distillation
LastOPD briefly aligns only the last-layer latent state when distilling Qwen3 into Qwen3-1.7B, beating token-only OPD on MATH-500.
1 stories, newest first · one card per story
LastOPD briefly aligns only the last-layer latent state when distilling Qwen3 into Qwen3-1.7B, beating token-only OPD on MATH-500.
Today 150 readers · 158 views · 27 reads · all time 199K views