Training-free spectral fingerprints match related LLM checkpoints from weights
Original title: Model-intrinsic, Weight-level, and Training-free Spectral Fingerprinting for Large Language Models
GATE singular-value features stay similar under instruction tuning, quantization, pruning, and distillation without prompts or forward passes.
A paper proposes model-intrinsic fingerprints built from singular-value distributions of GATE projection weights in gated feed-forward networks. Layer features are pooled by depth so models of different depths can be compared with linear centered kernel alignment. The pipeline needs no training, inputs, outputs, activations, or auxiliary verifier. Experiments on representative families and post-training variants separated source-related models from unrelated ones.
- Fingerprints come from spectral scalars of GATE weights, not query responses or activations.
- Depth-wise segmented pooling reduces the need for strict layer-to-layer alignment.
- Similarity stayed stable under instruction tuning, quantization, pruning, and distillation-related changes.
- The method is intended for ownership checks and offline auditing of open-weight checkpoints.
From the original article
1 min read at arxiv.orgThe increasing release and reuse of Large Language Models (LLMs) as open-weight checkpoints make model fingerprinting important for ownership protection and offline auditing. Existing fingerprinting methods typically construct evidence from query responses, generated outputs, or input-induced activations.
By Yanan He · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.