General VLMs rerank multimodal hits with one choice question
Original title: Multimodal Reranking Is a System One Decision
Zero-shot Qwen3.8-27B beats a supervised 2B reranker on three benchmarks and matches Reranker-8B on visual documents.
The paper argues a general vision-language model can rerank retrieval candidates as a System One decision, scoring one choice over the top 16 candidates in a single forward pass. On MMEB-V2 image retrieval, that choice scores 5.4 to 7.3 hit@1 above a yes/no question per candidate, which falls below the first stage. JEV-27B-VL, the same backbone tuned only on text decisions, improves page reranking by 0.6 to 1.1 points while leaving natural images unchanged.
Unconfirmed: The authors say the zero-shot model is larger and newer, so comparisons with supervised rerankers are confounded.
- Candidates are ranked by label probabilities from one forward pass.
- JEV-27B-VL scores 84.79 nDCG@5 versus 84.78 for Reranker-8B on visual documents.
- Reported latency is 1.1 to 1.5 times that of Reranker-8B.
- Page-reranking gains hold on most tasks, with a sign test p of at most 0.017.
From the original article
1 min read at huggingface.co
Multimodal rerankers are usually dedicated models: a vision-language model (VLM) is trained on retrieval data to score the candidates of a query. Scoring options is not specific to reranking. System One decision models such as Jev map a state and a typed question to one probability per option without generating text, and general models can make such decisions with little or no training.
By Zhuchenyang Liu, Yao Zhang, Yu Xiao · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.