To Align or Not To Align: Check Your COMPASS Before You Train
COMPASS predicts model-pair compatibility before alignment or merging, correlating 0.91 with cross-modal success.
COMPASS is a cheap diagnostic that predicts whether two pretrained models will work together before alignment or merging. It combines pair-specific k-NN representational overlap with a ceiling set by their training paradigms, then recommends whether to prioritize, invest more compute, change the pretraining paradigm, or deprioritize a pair. Across 72 cross-modal alignment experiments and 392 task-arithmetic merges, its estimates correlate with downstream success at average Pearson 0.91 and 0.70 respectively.
- COMPASS estimates compatibility before alignment or checkpoint merging.
- It uses k-NN overlap plus a paradigm-level agreement ceiling.
- Evaluated on 72 cross-modal alignments and 392 task-arithmetic merges.
- Pearson correlation is 0.91 for alignment and 0.70 for merging.
- Scores tell teams whether to align, add compute, switch paradigms, or skip.
From the original article
1 min read at huggingface.co
Combining pretrained models is a common strategy for building AI systems, whether by aligning a vision encoder with a language model or merging fine-tuned checkpoints. However, finding compatible pairs typically requires costly empirical search, as compatibility is known only after alignment or merging.
By Fabian Gröger, Petar Damjanović, Maria Brbić · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.