LoCaR retrieves localized image regions with frozen vision-language models
Original title: LoCaR: Localized Canvas Retrieval With Frozen Vision-Language Model
A Visual Genome-derived LOCFG benchmark targets smaller referents, and region encoding is the main reported gain.
LoCaR is a training-free framework for localized text-to-image retrieval that keeps vision-language encoders frozen. It uses SAM2 proposals to encode object regions independently, and studies an optional paired-support residual that does not update the encoders. The authors introduce LOCFG, derived from Visual Genome, for smaller and harder referents. Across several backbones, they report that better region representation is the main source of gain.
- SAM2 proposals focus encoding on individual objects.
- Paired-support adaptation is optional and does not update encoders.
- LOCFG is derived from Visual Genome.
- Results are reported across several vision-language backbones.
- AI models
- SAM2
From the original article
1 min read at arxiv.orgWhole image embeddings can under represent small objects and local attributes that distinguish relevant images for localized queries. We present LoCaR, a training free framework for localized text to image retrieval with frozen vision language models.
By Rahim Hossain · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.