Multi-Step Tool-Calling over Korean Open Public APIs: A Benchmark and a Data-Synthesis Recipe
Researchers introduce KOPA-Bench, a 145-task Korean public API tool-calling benchmark, and EDGE, an execution-grounded data synthesis method.
An arXiv paper presents KOPA-Bench, a benchmark of 145 real-world tasks chaining multiple tool-calls across live Korean government APIs, motivated by data-sovereignty requirements for on-premise open-source LLM agents. It also introduces EDGE, an execution-grounded dynamic graph that keeps only tool-output-to-input links verified by live API calls before synthesizing executable multi-step trajectories. A 9B model fine-tuned with GRPO on the resulting dataset nearly matches its untuned 27B family sibling on KOPA-Bench and improves on the BFCL benchmark.
32