DecisionRank on-policy distillation beats generative teacher on retrieval tasks
Original title: DecisionRank: On-Policy Distillation for Decision-Model Rerankers
Item-space OPD lets 1.7B DecisionRank surpass 27B LLM teacher on TREC DL while running 24.5x faster.
Decision models predict choices directly to avoid autoregressive latency. Transferring generative LLM ranking knowledge to decision models faces input-format mismatches. DecisionRank uses on-policy distillation where student scores form ranked lists fed to the teacher for score supervision. Combined with offline ranking loss, it reaches state-of-the-art on TREC DL, BEIR and BRIGHT.
- Outperforms 27B teacher at 1.7B parameters on TREC DL19-23
- Consistently beats alternatives across four benchmarks
- 24.5x faster than listwise counterpart on TREC DL
From the original article
1 min read at arxiv.orgDecision models target structured tasks by directly predicting choices or scores, avoiding autoregressive decoding latency. However, transferring a generative LLM's ranking knowledge to a decision model remains challenging: the teacher expresses preferences through a ranked list of document ID sequences, while the student determines the ranking using scores from the first-token position.
By Yanran Wu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.