Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreement
A preference-guided method adapts open-vocabulary segmentation using prompt disagreement, improving MESS results without pixel masks.
The paper adapts open-vocabulary semantic segmentation to specialized domains without dense pixel masks by treating disagreements among prompt templates as preference supervision. Localized queries are mined from high cross-template uncertainty and the model is updated with Region-Localized Preference Optimization plus consistency regularization outside those regions. On the MESS benchmark the method improves multiple OVSS backbones without pixel-level annotation and remains effective when preferences are noisy. Code is released at the authors' GitHub repository.
- Replaces dense mask labels with binary preferences from cross-template prompt disagreement.
- Mines localized queries in regions of high uncertainty between prompt templates.
- Adapts models with region-localized preference optimization and consistency regularization.
- Reports consistent MESS gains across OVSS backbones without pixel annotations, including noisy preferences.
- Products
- MESS
From the original article
1 min read at huggingface.co
Open-vocabulary semantic segmentation (OVSS) enables pixel-level prediction over arbitrary text-specified vocabularies and has shown strong generalization on common benchmarks. However, OVSS performance often degrades in specialized domains such as medical imaging, remote sensing, and industrial inspection, where dense pixel-level masks for adaptation are costly to obtain and require domain-specific expertise.
By Hyun-Kurl Jang, Jihun Kim, Kuk-Jin Yoon · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.