Your Prompt Should Do More: Effects of Retrieval Instructions in Embedding Models
Embedding models often ignore simple retrieval instructions when query-side distractors appear, but targeted fine-tuning largely fixes this.
Prompted embedding models are widely used for retrieval with detailed instructions, but they often fail to follow those instructions reliably. This paper studies how instructions change query representations in asymmetric retrieval and shows models can ignore even simple task instructions when query-side distractors are included. The authors link the failure to current training and evaluation setups. Fine-tuning with added query-side distractors substantially improves instruction following with minimal effect on other tasks.
- Models fail simple retrieval instructions when query-side distractors appear.
- Failures are linked to current embedding training and evaluation setups.
- Fine-tuning on query-side distractors improves following with little task impact.
From the original article
1 min read at arxiv.orgPrompted embedding models have recently received increasing attention, particularly for retrieval, where detailed retrieval instructions are provided as part of the retrieval prompt. Several new datasets and studies have examined this setting, showing that the current embedding models often struggle to follow such instructions reliably.
By Amanda Myntti · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.