Residual-space adaptation cuts forgetting in continual diffusion personalization
Original title: Continual Concept Personalization in Diffusion Models via Residual-Space Adaptation
Restricting updates to layers identified by residual dynamics keeps performance while cutting trainable parameters by up to 72%.
A paper extends residual-space adaptation to continual concept personalization in pretrained diffusion models, where new subjects, objects, or styles arrive sequentially. The method reports the highest concept fidelity among competing continual personalization methods and near-zero forgetting. Analysis of residual-space dynamics shows where adaptation capacity is concentrated. Limiting training to those layers preserves nearly the same performance and reduces trainable parameters by up to 72%.
- Concepts are learned from only a few reference images.
- Existing methods typically train concepts independently or all at once.
- The setting covers sequential subjects, objects, and styles.
- Parameter cuts of up to 72% come from restricting adaptation to identified layers.
From the original article
1 min read at arxiv.orgText-to-image personalization aims to equip pretrained diffusion models with the ability to learn user-specific concepts from only a few reference images. Existing methods, however, typically train each concept independently or assume that all concepts are available simultaneously.
By Yuzhi Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.