Fingerprinting Multimodal Large Language Models
Researchers introduce AttnPrint and DistillTrace, fingerprinting and black-box auditing methods that detect multimodal LLM derivative models and unauthorized distillation across 19 architectures.
The paper presents the first study on multimodal model fingerprinting, targeting illicit deployment and unauthorized distillation of multimodal large language models. AttnPrint extracts cross-modal attention distributions and isolates their low-frequency components as white-box fingerprints, while DistillTrace applies hypothesis testing on model outputs for black-box distillation auditing. Experiments cover 154 model instances across 19 multimodal architectures, showing strong derivative-model detection robust to five downstream modification techniques and evidence of distillation relationships under three parameter-independent techniques.
- First study on multimodal LLM fingerprinting, addressing illicit deployment and unauthorized distillation detection.
- AttnPrint uses low-frequency components of cross-modal attention distributions as white-box fingerprints.
- DistillTrace applies hypothesis testing on outputs for black-box distillation auditing.
- Evaluated on 154 model instances across 19 multimodal architectures, robust to five modification techniques.
Full article163 words · extracted from arxiv.org · click to collapse
While multimodal large language models (MLLMs) enable a wide range of image-text reasoning tasks, recent incidents indicate that they are vulnerable to illicit deployment and unauthorized distillation. Existing solutions for model provenance are typically confounded by shared language backbones in MLLMs and struggle to detect violations of distillation. To bridge this gap and safeguard model ownership, we present the first study on multimodal model fingerprinting. Inspired by recent findings that self-attention acts as a low-pass filter and that its low-frequency components are informative, we develop AttnPrint for white-box provenance. Specifically, we extract cross-modal attention distributions and isolate their low-frequency components to serve as model fingerprints. To facilitate black-box auditing, we further introduce DistillTrace, which employs hypothesis testing of MLLM outputs to identify potential model infringement. We conduct extensive experiments on 154 model instances across 19 multimodal architectures. Notably, AttnPrint achieves strong derivative-model detection performance while remaining robust to five downstream modification techniques. DistillTrace also provides evidence of distillation relationships under three parameter-independent techniques.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.20457