Privileged-information design shapes on-policy self-distillation gains
Original title: Predicting Privileged Information Effectiveness for On-Policy Self-Distillation
An error-step KL contrast predicted training gains better than a utility score that can leak information.
The paper argues that privileged information for on-policy self-distillation is not a fixed design choice, especially in multi-turn agent trajectories. Effectiveness depended on content, generation source, and the student model. Denser trajectories and information from more capable generators generally produced larger gains, while thinking models benefited less consistently. The authors say selected privileged information still helped when combined with GRPO.
Unconfirmed: Gain rankings are the authors' measurements; independent replication is not reported.
- The study compared 16 privileged-information configurations.
- Tests used three multi-turn agentic benchmarks and three student models.
- Selecting each task's information by error-step KL contrast beat random selection.
- Selection gains were largest when privileged-information candidates differed widely.
- With GRPO, effective privileged information beat a strong GRPO baseline, the authors report.
From the original article
1 min read at huggingface.co
On-policy self-distillation (OPSD) enables large language models (LLMs) to learn from privileged information (PI) without a separate teacher model. However, prior work largely treats PI as a fixed design choice, leaving the effects of its content and construction poorly understood.
By Kaijian Zou, Emiliano Penaloza, MohammadHossein Rezaei · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.