InstantFusion: A Shared Latent Interface for Heterogeneous Image Generators
InstantFusion connects heterogeneous image generators through a shared latent denoising interface.
InstantFusion adds model-specific Latent AutoEncoders that project intermediate states from heterogeneous diffusion or flow image generators into a shared latent space and decode them for a target generator. The shared space aligns states derived from the same image at matched noise levels and synchronizes local denoising velocities. After training, denoising can move across frozen generators without a dedicated mapping for every model pair. Demonstrations cover acceleration, multi-preference composition, cross-architecture on-policy distillation, and cross-model composition.
- Independently trained generators show similar latent spatial organization.
- Model-specific Latent AutoEncoders map states into a shared space.
- Training aligns matched noise levels and local denoising velocities.
- Frozen generators can be composed without pairwise mappings.
From the original article
1 min read at huggingface.co
Diffusion/flow models have emerged as the de facto paradigms for image generation, giving rise to a diverse family of modern generators with increasingly heterogeneous architectures and representations. Despite their remarkable generation capabilities, the cross-model alignment of latent representations and the consistency of their underlying denoising dynamics remain underexplored.
By Xuanlang Dai, Jiazi Bu, Yujie Zhou · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.