SynthAPI scaling law links tool-use gains to environment diversity
Original title: Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis
Across 458,752 tasks and 849 models, tool and database diversity beat adding tasks as data budgets grow.
The SynthAPI pipeline controls diversity of synthetic tool-use environments along tools, tasks, and the databases those tools use. Varying those controls across four model sizes produces a scaling law for downstream generalization. The authors report that agents trained under these findings beat prior synthetic tool-use data on four evaluation benchmarks.
- The study trained 849 models of four different sizes.
- Larger data budgets favor tool and database diversity over more tasks.
- Raising data complexity produced larger gains than simpler synthetic data.
- Results are compared with prior tool-use agent and environment-generation data.
From the original article
1 min read at arxiv.orgWe explore how to build performant large language model (LLM) tool-use agents that complete complex, multi-step tasks. Specifically, we study how data composition in synthetically generated tool-use environments affects the training of such agents.
By Andrew Szot · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.