EDGEGEN: Improving Tool-Calling Agents Beyond Happy Paths with Synthetic Edge Case Generation
EdgeGen synthesizes database-grounded edge-case tasks that improve tool-calling agents without human annotation.
EdgeGen extracts compliance rules from a tool-calling agent's specification and generates database-grounded tasks designed to violate those rules, addressing generic synthetic data that ignores agent state. Combined with existing generators, it supports finetuning and harness optimization in a fully automated loop with no human annotation. Finetuning produces mean progress gains of 2 to 42 percent on the tau2-bench airline domain, while some baselines degrade certain models. For Gemma-4-e4b, harness optimization improves mean progress about 10 percent over a human-curated harness and 30 percent over the base harness.
- Compliance rules from the agent spec drive database-grounded violating tasks.
- The closed loop combines generation, finetuning, and harness optimization without labels.
- Finetuning improves mean progress 2 to 42 percent on tau2-bench airline.
- Gemma-4-e4b harness gains are 10 percent over human-curated and 30 percent over base.
- Products
- EdgeGentau2-bench
- Organizations
- Google DeepMind
- AI models
- Gemma-4-e4b
From the original article
1 min read at huggingface.co
Tool-calling LLM agents are increasingly deployed in enterprise applications. However, effective evaluation and optimization require high-quality, diverse task datasets that are often difficult to obtain due to privacy and other constraints. Existing synthetic task generation methods often produce generic tasks that ignore an agent's underlying state or database and fail to reflect real-world usage diversity.
By Harshavardhan Abichandani, Penny Chong, Jiyuan Shen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.