SAGE: Salient Factor Discovery and Generation with Visual Foundation Representations
SAGE discovers unlabeled visual subtypes in a frozen latent and conditions a diffusion transformer to generate them.
SAGE learns target-specific salient factors and shared content in the high-dimensional spatial latent of a frozen representation autoencoder, then conditions a diffusion transformer on a reference image's salient representation. On Digits-ImageNet and FFHQ eyeglasses it reports reconstruction rFID below 2 and unsupervised subtype discovery, with digit probe accuracy of 0.950 versus at most 0.281 for baselines. Salient-conditioned generation raises Digits-ImageNet subtype accuracy from 27.7% to 90.5% and increases diversity. On retinal OCT, the salient space separates three diseases using only normal-versus-disease labels.
- SAGE factors salient detail inside a frozen autoencoder's spatial latent.
- Digit probe accuracy reaches 0.950 versus at most 0.281 for baselines.
- Conditioned generation lifts subtype accuracy from 27.7% to 90.5%.
- Retinal OCT salient space separates three diseases without subtype labels.
From the original article
1 min read at huggingface.co
Given a target dataset, such as faces with eyeglasses, and a background dataset, such as faces without, contrastive analysis separates salient factors specific to the target from common content shared by both.
By Shuang Liang, Lejun Liao, Shiyuan Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.