NitroFusion2 brings adversarial loss to the text stream of multimodal diffusion transformers
Original title: NitroFusion2: Bringing Adversarial Loss to the Text Stream of Multimodal Diffusion Transformers
NitroFusion2 applies adversarial distillation to the text stream of MMDiTs, enabling one-step generation of sharp text and details.
The work introduces Text-Stream Adversarial Distillation and Aspect-Cued Discrimination for multimodal diffusion transformers. It builds on NitroFusion to bring few-step distillation down to one or two steps. The text stream provides a steerable representation for adversarial training.
- It produces sharp rendered text and rich fine-grained details at one step.
- It remains competitive at two steps.
From the original article
1 min read at huggingface.co
Adversarial diffusion distillation has conventionally relied on image features for discrimination. We revisit this choice for multimodal diffusion transformers (MMDiTs), whose text stream becomes image-dependent through joint attention. We introduce Text-Stream Adversarial Distillation (TSAD), which exploits intermediate text-token states as a new adversarial representation.
By Dar-Yen Chen, Yi-Zhe Song · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.