Embedding search finds 92% of trial evidence versus 45% for labels
Original title: Nothing Left Behind: Finding Evidence from a Large Randomized Evaluation Corpus
On 7,354 randomized trials, a two-stage embedding-plus-language-model filter recovers far more systematic-review evidence, with cost rising by search depth.
Researchers compare curated label search with an embedding-based pipeline that retrieves candidates by meaning and then uses a language model to keep only relevant items. In a corpus of 7,354 randomized trials, label search found 45% of the evidence a systematic review would include, while the new approach found 92%. Cost per query rises with search depth. The authors argue for broad candidate retrieval followed by query-specific evidence filtering.
Unconfirmed: Recall figures are the authors' reported results; no independent replication is stated.
- The evaluation corpus contains 7,354 randomized trials.
- Label search recovered 45% of systematic-review evidence.
- The embedding-plus-filter approach recovered 92%.
- Per-query cost increases as search depth increases.
From the original article
1 min read at arxiv.orgMany AI systems need to find every relevant items in a large collection and then organize what they found. The standard way to do this is to tag each item with labels from a curated vocabulary and search on those labels. Those labels are useful for organizing results, but searching on them misses items that do not fit a label and returns too much when labels are broad.
By Giuliano Martinelli · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.