Beyond Spatio-Temporal Priors: A Generalizable Approach for Dense Correspondence Matching
FreeMatching matches dense correspondences when image edits break classical motion and geometry priors.
FreeMatching targets identity-preserving dense correspondence for image editing and reference-guided generation, where smooth-motion and rigid-geometry priors fail. It combines generative and semantic foundation representations with heterogeneous supervision from classical datasets, tracked videos, and synthetic scenes. Teacher-guided iterative refinement improves correspondence on editing pairs without dense correspondence annotations. A single model improves challenging editing pairs, stays competitive on classical benchmarks, and can score identity preservation in line with human judgment. Code is available on GitHub.
- Drops smooth-motion and rigid-geometry assumptions for image-editing tasks.
- Combines classical, tracked-video, and synthetic supervision.
- Teacher-guided refinement needs no dense editing annotations.
- Identity-preservation scores correlate with human judgment.
- AI models
- FreeMatching
From the original article
1 min read at huggingface.co
Dense correspondence matching has historically been bounded by simplifying spatio-temporal priors, such as smooth motion and rigid geometry. While effective for classical tasks, these assumptions break down in image editing and reference-guided generation (IEG), where transformations can preserve visual identity while breaking physical continuity.
By Luping Liu, Bingyi Kang, Yifan Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.