Dense retrievers favor passages naming high-scoring answer objects
Original title: Search-by-Answer Bias in Retrieval Models
Across eight models, higher token-prediction scores predict higher retrieval ranks even when the relation is wrong.
The authors report search-by-answer bias in dense retrievers: passages that mention answer objects the model scores as likely outrank passages supporting valid alternatives. Controlled factual contrasts from LAMA/TREx link a token-prediction head's object scores to passage rankings. The pattern holds for DPR, Spider variants, Contriever, E5, and Qwen3-Embedding, including when passages differ only in the answer object.
- The bias remains when the high-scoring object is paired with a wrong relation.
- Tests use controlled factual contrasts drawn from LAMA/TREx.
- Larger token-prediction scores consistently predict higher retrieval scores.
- AI models
- DPRContrieverE5Qwen3-Embedding
From the original article
1 min read at arxiv.orgDense retrievers are meant to select passages that match a query's semantic specification. We show they suffer from search-by-answer bias: they favor passages that mention answer objects the model knows (scores as likely), over passages that support valid alternative answers. We test this with controlled factual contrasts from LAMA/TREx.
By Adiel Avigad · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.