Disentangling Paradigm, Identifier, and Decoding in Generative Retrieval
Decoding choice, not just the paradigm, shifts diffusion generative retrieval Hit@1 by up to 13.7 points.
The paper compares autoregressive, masked-diffusion, and block-diffusion generative retrievers on NQ320K and MS300K using residual-quantised, product-quantised, and random identifiers. With identifier length and training budget fixed, decoding alone changes a diffusion model's Hit@1 by 6.6 to 13.7 points. One-pass scoring matches or beats generate-and-match in 11 of 12 settings, while autoregressive models still lead Hit@1. On NQ320K, random identifiers keep 83-90% of residual-quantised Hit@1, indicating substantial query-to-identifier memorization.
- Decoding alone moves diffusion Hit@1 by 6.6 to 13.7 points.
- One-pass scoring matches or beats generate-and-match in 11 of 12 settings.
- Autoregressive models still lead Hit@1; on NQ320K the model, not beam search, drives the lead.
- Random identifiers retain 83-90% of residual-quantised Hit@1 on NQ320K.
From the original article
1 min read at arxiv.orgGenerative retrieval trains a language model to generate the identifier of a relevant document. Recent work replaces the autoregressive decoder with diffusion, but changes identifiers, training recipe and decoding at once, so differences cannot be credited to the paradigm.
By Hicham Randrianarivo · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.