Black-box signatures proposed to detect LoRA adaptations of diffusion models
Original title: Provenance Signatures for Low Rank Adaptations of Image Diffusion Models
Tests use only the intrinsic dimension of generated images and assume unknown adaptations and model families.
The paper defines a provenance protocol for telling whether an image service uses a base diffusion model, a Low Rank Adaptation of that model, or a different model family. The setting is an open world in which candidate adaptations and alternative models are unknown, unlike prior identification against a fixed list. Signatures are black-box and based on the intrinsic dimension of sets of generated images, using hypothesis testing. The stated motivation is that open base models can be adapted to produce biased, harmful, or illegal images.
Unconfirmed: Detection performance figures are not given in the abstract, so practical accuracy is unstated.
- Two problems are defined for multiple model families and multiple adaptations.
- The method does not require a list of known candidate models.
- Signatures depend only on intrinsic dimension of generated image sets.
- The protocol is intended to separate base models, LoRA adaptations, and other families.
From the original article
1 min read at arxiv.orgDiffusion models have revolutionized the generation of realistic artificial images. Efficient techniques such as Low Rank Adaptation (LoRA) can modify these models for particular use cases, making them more diverse and widely usable. At the same time, open base models may be adapted to deliver biased, harmful or illegal images in an online service.
By Tobias Cheung · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.