ToolLoop: Closed-Loop Tool-Use Data Synthesis via Decomposed Generation and Dynamic Self-Feedback
ToolLoop introduces a closed-loop synthetic data framework whose 11K examples lift a 4B model to 86.40% on BFCL tool-use evaluation.
ToolLoop decomposes tool-use data synthesis into function-name sampling, backward derivation of user queries, and forward derivation of tool calls, with dynamic self-feedback at each stage. This shifts the paradigm from generate-then-filter to generate-verify-refine, reducing inefficient and imbalanced synthetic data. A 4B model trained on 11K synthetic examples reaches 86.40% accuracy on BFCL non-reasoning mode (86.07% in an Isolate variant excluding BFCL-overlapping functions) and 72.1% on ACEBench using only 18.3% of baseline training data.
- Closed-loop generate-verify-refine replaces static generate-then-filter synthesis
- Three stages: function-name sampling, backward query derivation, forward tool-call derivation
- 4B model reaches 86.40% BFCL non-reasoning with 11K synthetic examples
- 72.1% ACEBench accuracy using only 18.3% of baseline training data
- Products
- ToolLoop
- Organizations
- Berkeley Function Calling LeaderboardACEBench
From the original article
1 min read at arxiv.orgHigh-quality tool-use data is critical for training language models to interact effectively with external tools. However, existing synthetic approaches typically follow a generate-then-filter paradigm with static post-hoc verification, often yielding inefficient data with imbalanced feature distributions.
By Min Zeng · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.