Structured Residual Connectivity Matters for Diffusion Transformers
Structured residual paths in diffusion transformers cut FID and training iterations with almost no extra parameters.
The paper redesigns residual connections in Diffusion Transformers so each block can selectively retrieve earlier spatial and semantic features instead of summing a single residual stream. The structured local and long-range paths add under 0.1% parameters and cut training iterations by up to 1.73×. Applied to REPA-XL/2, FID falls from 5.9 to 4.34 without guidance and reaches 1.39 with classifier-free guidance.
- DiTs use a uniform residual stream unlike U-Net skip connections.
- Structured local and long-range paths let blocks retrieve earlier features.
- Training needs up to 1.73× fewer iterations with under 0.1% extra parameters.
- REPA-XL/2 improves from 5.9 to 4.34 FID, and 1.39 FID with guidance.
- AI models
- REPA-XL/2
From the original article
1 min read at huggingface.coDiffusion Transformers (DiTs) have established themselves as a scalable backbone for high-fidelity image synthesis. However, unlike U-Net based diffusion models that rely on rigid, hand-crafted skip connections, DiTs predominantly use a uniform residual stream that integrates all preceding layers as a monolithic state.
By Yuhe Liu, Xinyin Ma, Gongfan Fang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.