Turn-level best-of-N sampling improves LLM agent accuracy by 5-10%
Original title: On Turn-Level Best-of-N Sampling for Scaling LLM Agents
Turn-level best-of-N sampling with generator-selector LLMs yields 5-10% accuracy gains and improved consistency on long-horizon tasks by selecting optimal actions at each step.
LLM agents struggle with complex long-horizon tasks during inference. Turn-level best-of-N sampling generates N candidate actions per turn with a generator model then selects one using a selector model. Experiments using off-the-shelf open-weight LLMs show significant performance improvements in accuracy and consistency. Trajectory analysis reveals shifts toward better instruction following, debugging and testing.
- Generator produces N candidate actions per turn
- Selector model chooses one executed action
- 5-10% accuracy improvement on long-horizon tasks
- Improved consistency across repeated attempts
- Analysis shows reduced attempts and better task following
From the original article
1 min read at arxiv.orgIn this work, we evaluate a simple inference-time scaling approach based on \textit{turn-level best-of-N sampling} (\method) for large language model (LLM) agents on complex, long-horizon tasks.
By Jalaj Bhandari · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.