Embedding Prediction Helps Image Generation
NEPA-DiT-XL predicts embeddings each denoising step and reaches ImageNet FID 1.32 with about one-third of REPA's compute.
Next-Embedding Predictive Autoregression trains a Transformer to predict the next continuous embedding rather than reuse one fixed class or text condition. In Embedding Conditioned Generation, a diffusion transformer is conditioned on Multi-Embedding Prediction outputs that are recomputed at every denoising step. On class-conditional ImageNet 256x256, NEPA-DiT-XL combined with REPA reaches an FID of 1.32 while using about one-third of REPA's training compute, at the cost of a second network on each sampling step.
- NEPA predicts next continuous embeddings instead of reusing a fixed class or text condition.
- Multi-Embedding Prediction forecasts clean-image embeddings from the condition and noisy image.
- The diffusion generator is reconditioned on those predictions at every denoising step.
- NEPA-DiT-XL reaches FID 1.32 on ImageNet 256 using about one-third of REPA compute.
From the original article
1 min read at arxiv.orgIn diffusion transformers, a class label or a text prompt is embedded once, and the same condition is reused at every denoising step. We ask whether predicted embeddings can serve as this condition instead. Next-Embedding Predictive Autoregression (NEPA) trains a Transformer to predict the next continuous embedding in a sequence.
By Sihan Xu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.