MFlow learns when pixel diffusion should predict images or velocity
Original title: Rethinking the design space of Vanilla Pixel Diffusion Transformers
A post-hoc ConvRefiner, trained with the backbone frozen, restores high-frequency image statistics while preserving FID.
Researchers revisit vanilla pixel-space diffusion transformers and introduce MFlow, which learns to interpolate between clean-image and velocity prediction across principal data directions. Preference shifts toward velocity prediction as model capacity grows. They report competitive or improved results on ImageNet, UCF-101 video, and large-scale text-to-image generation, and add a lightweight convolutional module trained afterward with an adversarial loss.
- JiT used clean-image prediction so a vanilla diffusion transformer could generate in pixel space.
- MFlow interpolates between x0-prediction and velocity prediction during diffusion training.
- Higher model capacity shifted the learned preference toward velocity prediction.
- Evaluations cover ImageNet, UCF-101 video, and large-scale text-to-image generation.
- Pixel diffusion transformers still showed a high-frequency deficit versus real images.
From the original article
1 min read at arxiv.orgPixel-space diffusion models have recently re-emerged as a compelling alternative to latent diffusion, avoiding the lossy latent bottleneck and separately trained autoencoder. JiT shows that a vanilla diffusion transformer can generate directly in pixel space by replacing the commonly used velocity prediction ($\boldsymbol{u}$-prediction) with clean image prediction ($\boldsymbol{x_0}$-prediction).
By Soumik Mukhopadhyay · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.