Swift trains stronger diffusion models with a frozen weak denoiser
Original title: SWIFT: Self-guided Weak-to-strong dIFfusion models via Target extrapolation
On one ImageNet-256 setup it beat the baseline's best FID with about half the training compute, including guide overhead.
Swift extrapolates the clean regression target away from a frozen weak denoiser so a stronger model emphasizes underfit components, without extra inference cost. The weak guide can be an early checkpoint, needs no backpropagation during main training, and is discarded afterward. The resulting model keeps the original sampler and supports conditional and unconditional generation without label dropout. Experiments on CIFAR-10 and ImageNet-256 report gains across scales.
- A nonempty extrapolation range is proved to lower posterior-mean error versus matched training.
- One ImageNet-256 run used roughly half the baseline's total training compute.
- On RAEv2, FID was 1.15 without inference-time guidance and 0.99 with guidance.
- The weak guide is discarded after training and adds no inference overhead.
From the original article
1 min read at huggingface.co
Training high-quality diffusion models is computationally expensive, and inference-time guidance often adds further sampling cost. In this work, we introduce Swift, a simple yet effective self-guided training method that uses a frozen weak denoiser to improve a stronger model without adding inference overhead.
By Yanjin He, Cuifeng Shen, Wanzhou Lei · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.