Understanding On-Policy Distillation: A Mechanistic Interpretability Perspective via Sparse Crosscoders
Sparse crosscoders show on-policy distillation reweights features LLMs already share rather than creating new ones.
The paper uses sparse crosscoders to ask what on-policy distillation changes inside an LLM student. A proposed swap readout shows OPD neither creates features nor transfers the teacher's own features, and leaves firing rates of over 98% of frequently used student features within 20%. The usual SFT warm-up on teacher rollouts reweights shared features, including conversation format, reasoning style, and mathematical notation, and those shifts persist through OPD. Replaying that reweighting without changing weights nearly matches the warmed-up student's accuracy, while shuffled features do not.
- Sparse crosscoders share one feature dictionary across student and teacher.
- Swap readout measures how each checkpoint uses features.
- OPD creates no new features and does not copy the teacher's.
- Over 98% of frequent student features stay within 20% firing rate.
- SFT warm-up on teacher rollouts reweights shared features before OPD.
From the original article
1 min read at huggingface.co
On-policy distillation (OPD) is a widely adopted post-training technique for LLM reasoning. It is commonly believed to transfer knowledge from a stronger teacher, yet what OPD actually distills into the student's internal representations remains unclear. We study this question with sparse crosscoders, which learn one feature dictionary shared by the student before and after OPD and the teacher.
By Zichao Yu, Qianshuo Ye, Xu Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.