Learning Representations to Guide Pixel Diffusion Transformers
Representation Guidance lets a 459M pixel diffusion model beat larger transformers, with FID 1.55.
The paper finds that better external visual representations do not automatically help pixel diffusion transformers, because aligned semantics can be lost during synthesis. Representation Guidance forces generation-relevant information through a low-dimensional bottleneck and uses the induced prediction offset to modify the flow-matching target. RG-L/16, with 459M parameters, outperforms much larger pixel diffusion transformers; RG-XL/16 reaches FID 1.55 on ImageNet 256x256, FID 1.74 at 512x512, 0.89 on GenEval, and 87.1 on DPG-Bench.
- Stronger external semantics do not automatically improve pixel generation.
- RG uses a bottleneck and flow-matching offset inside a plain Transformer.
- RG-L/16 has 459M parameters and beats larger pixel diffusion transformers.
- RG-XL/16 reaches FID 1.55; 512px FID is 1.74; GenEval is 0.89.
From the original article
1 min read at huggingface.co
Visual representations are largely developed for understanding, yet what makes them useful for generation remains unclear. We study this question in pixel diffusion transformers and uncover two key insights. First, better representation quality does not necessarily translate into greater generative utility, as externally aligned semantics can be progressively lost during pixel synthesis.
By Zihao Pan, Mengting Chen, Cheng Shi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.