Study finds nearest-neighbor voting beats LLMs on tabular ICL
Original title: Beyond the Vote: When Do LLMs Add Value to Retrieval-Based Tabular Classification?
Across 19 datasets, Distance-KNN had the highest mean accuracy at every pool size, with LLM gains only under low label consensus.
A paper compares four general-purpose LLMs with distance-weighted nearest-neighbor voting using identical retrieved neighbors on 19 tabular classification datasets. With the number of demonstrations fixed, larger retrieval pools improved every predictor, but Distance-KNN had the highest mean accuracy at every pool size. LLM gains were concentrated where neighbors showed low label consensus and depended on distance to the query; all LLMs lagged on average in the most distant group.
Unconfirmed: The four LLMs are not named in the abstract.
- Some LLMs beat Distance-KNN in close and intermediate neighborhoods when label consensus is low.
- Tests of demonstration count, prompts, retrieval representation, and label balance kept the pattern.
- The abstract does not name the four general-purpose LLMs.
From the original article
1 min read at arxiv.orgRetrieval-based in-context learning (ICL) enables large language models (LLMs) to predict tabular labels from similar examples, but it remains unclear whether LLM inference improves on direct voting over those same examples.
By Ziquan Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.