Continually Post-training Small Models via Synthetic Self-distillation
SynthSelf lets small models continually post-train by outsourcing in-distribution rewrites to a slightly larger same-family aligner.
The paper introduces Synthetic Self-Distillation (SynthSelf) so smaller models can continually post-train without the strong in-context learning that Self-Distillation Fine-Tuning requires. A minimally larger same-family aligner produces in-distribution rewrites, combined with off-policy supervision and rejection sampling. Across 4 acquisition datasets, including a mixture, and 14 retention benchmarks, SynthSelf beat 6 baselines at 3 small-model scales. The authors link acquisition failure and forgetting to distribution shift after post-training.
- SDFT fails on small models that lack strong in-context learning.
- Forgetting and poor acquisition track post-training distribution shift.
- A slightly larger same-family model writes in-distribution supervision.
- SynthSelf beat six methods on four acquisition sets and 14 retention benchmarks.
From the original article
1 min read at huggingface.co
Continual post-training faces a dual challenge of acquisition and retention: models must learn new capabilities while preserving prior knowledge. While Self-Distillation Fine-Tuning (SDFT) mitigates this using self-supervision with in-context demonstrations, it relies on strong in-context learning and instruction-following abilities. Consequently, SDFT fails on smaller models critical for low-latency, on-device use.
By Woosung Koh, Hyunsoo Lee, Kyungjae Lee · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.