Data-free method distills diffusion models to one-step generation
Original title: Consistent Distribution Matching for Data-Free Diffusion Distillation
Authors report ImageNet 256 FID of 2.04 at 1-NFE and 1.37 at 4-NFE after 40 epochs, without distillation data.
Researchers propose Consistent Distribution Matching, a simulation-free, data-free way to distill flow and diffusion models into one-step or few-step generators. A frozen teacher and one trainable student share a single objective that unifies sampling and score estimation, avoiding curated datasets, teacher rollouts, and proxy networks. They prove the objective implies Wasserstein convergence of student pushforwards to teacher marginals. Code and a model are linked.
Unconfirmed: FID figures are author-reported and not independently verified here.
- The setup uses only a frozen teacher and a trainable student.
- On ImageNet 256 the method reports 1-NFE FID 2.04.
- A 4-NFE FID of 1.37 is reported within 40 epochs.
- Authors say it beats data-free distillation baselines.
- Code and model are at consistentdmd.github.io.
From the original article
1 min read at huggingface.co
Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one-step or few-step generation. However, existing methods often depend on curated distillation datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate model training and scaling.
By Yuxiang Fu, Qi Yan, Zike Wu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.