Fingerprinting Multimodal Large Language Models
Researchers introduce AttnPrint and DistillTrace, fingerprinting and black-box auditing methods that detect multimodal LLM derivative models and unauthorized distillation across 19 architectures.
The paper presents the first study on multimodal model fingerprinting, targeting illicit deployment and unauthorized distillation of multimodal large language models. AttnPrint extracts cross-modal attention distributions and isolates their low-frequency components as white-box fingerprints, while DistillTrace applies hypothesis testing on model outputs for black-box distillation auditing. Experiments cover 154 model instances across 19 multimodal architectures, showing strong derivative-model detection robust to five downstream modification techniques and evidence of distillation relationships under three parameter-independent techniques.
- First study on multimodal LLM fingerprinting, addressing illicit deployment and unauthorized distillation detection.
- AttnPrint uses low-frequency components of cross-modal attention distributions as white-box fingerprints.
- DistillTrace applies hypothesis testing on outputs for black-box distillation auditing.
- Evaluated on 154 model instances across 19 multimodal architectures, robust to five modification techniques.
- Products
- AttnPrintDistillTrace
From the original article
1 min read at arxiv.orgWhile multimodal large language models (MLLMs) enable a wide range of image-text reasoning tasks, recent incidents indicate that they are vulnerable to illicit deployment and unauthorized distillation. Existing solutions for model provenance are typically confounded by shared language backbones in MLLMs and struggle to detect violations of distillation.
By Chao Huang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.