What Does Privileged Information Add to On-Policy Self-Distillation?
Study of on-policy self-distillation on 5,319 AMPLE-Math problems finds reference-free distillation drives most gains, with modest extra benefit from privileged references.
30
Study of on-policy self-distillation on 5,319 AMPLE-Math problems finds reference-free distillation drives most gains, with modest extra benefit from privileged references.
Entities are extracted by the model from each article. First seen , last mentioned . Watching keeps it in this browser only (no account); the watchlist page and dashboard alerts use it.
Today 1.6K readers · 2.5K views · 792 reads · all time 189K views