Parallel agent teams scale test-time compute better
Original title: Test-Time Compute Scales Better with Parallel Agent Teams
Parallel teams of ten outperform best-of-N and large single teams on long-horizon tasks with a 27B model
The paper proposes parallel agent teams that sample many small teams in parallel and keep the best result. On 12 long-horizon tasks, teams of ten with a 27B model beat best-of-N sampling on 10 tasks and beat one large team of 100 on four tasks. At equal compute cost they beat a frontier model on half the tasks. The method balances high mean performance with variance gains.
- Teams of ten beat best-of-N on 10 of 12 tasks
- Beat large team of 100 on all four tested tasks
- Balances mean and variance for improved scaling
- Uses 100-agent total budget
- AI models
- 27B model
From the original article
1 min read at arxiv.orgTest-time scaling improves the performance of a fixed model by spending more computation at inference. There are two common ways to spend it on agents when a verifier can score every attempt. Best-of-N sampling runs many independent agents and keeps the best result but saturates quickly.
By Junlin Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.