LASAGNA finds RAG component gains often fail to compose
Original title: Retrieval Augmented Generation Compositionality
Three techniques predicted a 47% combined gain but delivered 17% together across 450 configurations.
The paper introduces LASAGNA, an open-source framework for controlled evaluation of composed retrieval-augmented generation pipelines. It pre-declares three hypotheses and tests 450 configurations on eight benchmarks with corpora from 5K to 200K documents. Individually strong techniques did not reliably compose: three retrieval methods predicted a combined 47% improvement but delivered 17% together. Retrieval quality and answer faithfulness favored different choices, and the best composition changed as corpus size grew.
Unconfirmed: The reported percentage gains are the authors' controlled experiments, not field measurements.
- LASAGNA tests embedding, chunking, retrieval, reranking, and related stages together.
- Corpora range from 5K to 200K documents across eight benchmarks.
- Optimizing retrieval can reduce answer faithfulness, and the reverse also holds.
- The winning composition on a small corpus may lose at larger scale.
From the original article
1 min read at huggingface.co
Retrieval Augmented Generation (RAG) grounds large language models in external knowledge. Modern pipelines combine embedding, chunking, retrieval, reranking, query augmentation, context compression, and agentic verification, yet these techniques are typically evaluated one at a time.
By Ido Levy, Segev Shlomov, Leshem Choshen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.