Can We Trust the Teacher? Decoupled Credit Direction-Magnitude for Self-Distillation
A new self-distillation method separates credit direction from magnitude and beats prior RL baselines.
The paper argues that updating step-level credit direction and magnitude together under teacher supervision leaves both exposed to teacher errors and preference variance. Decoupled Credit Self-Distillation (DCSD) uses belief-margin probing for direction and marginal information gain for magnitude, then calibrates privileged teacher supervision for step-to-token credit assignment. Across 11 benchmarks it scored best against GRPO, OPSD, RLSD, and RLCSD, improving base models by 8.45 points on mathematical reasoning and 7.01 on multimodal reasoning, while correcting credit direction for 6% of tokens and reducing token credit magnitude by 1.5 times.
- DCSD separates credit direction from contribution magnitude.
- Belief-margin probing sets direction; information gain sets magnitude.
- It beat GRPO, OPSD, RLSD, and RLCSD across 11 benchmarks.
- Gains were 8.45 math points and 7.01 multimodal points over base models.
From the original article
1 min read at huggingface.co
RLVR provides reliable trajectory-level credit, while OPSD offers dense supervision for token-level credit.
By Yugu Li, Zehong Cao, Peizhen Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.