Multi-agent Scaling Across Disjunctive and Compensatory Tasks
Larger LLM teams rarely help: plurality voting wastes a 5–20 point coverage gain, and shared bias dominates Fermi error.
Using Steiner's taxonomy, the paper studies multi-agent LLM scaling on disjunctive and compensatory tasks across 13 open-weight models and teams of up to 30 agents. The chance that at least one agent is correct grows 5–20 points with team size, but plurality voting realizes almost none of that gain, matching theory within 0.5 points. Multi-round revision helps, yet one peer yields nearly the same gain as 29. On Fermi estimation, shared item-level bias accounts for about 87% of squared error, so averaging reduces error by only about 6%.
- At-least-one correctness rises 5–20 points as teams grow to 30
- Plurality voting captures almost none of that potential
- One revising peer nearly matches gains from 29 peers
- Shared bias is about 87% of Fermi squared error
- Averaging cuts Fermi error by only about 6%
From the original article
1 min read at arxiv.orgMulti-agent LLM systems are often expected to improve as team size increases, yet the scaling behavior may depend on task structure. Our central contribution is to introduce Steiner's taxonomy of group tasks as a framework for analyzing multi-agent LLM scaling and focusing the analysis on disjunctive and compensatory tasks.
By Carolina Fortuna · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.