ReTrace compiles agent SOP runs into code-selected workflows
Original title: ReTrace: Distilling Tool-Using Agents into Planner-Free Workflows
On 79 production procedures, compiled workflows reached 93.8% completion versus 96.8% for the agent at 24% lower cost.
ReTrace turns a natural-language SOP and a few logged agent runs into a reusable workflow where code, not an LLM, chooses the next step. It extracts conditions from the agent's written reasons for tool calls, merges them into a rule document, and compiles that document for different orchestration platforms. On production procedures it nearly matches agent completion at lower cost. Benchmark results show large latency cuts, with repair from labeled failures improving success on τ²-Bench and SOP-Bench.
Unconfirmed: Completion and cost figures are from the authors' platform and benchmarks, not an independent audit.
- On τ²-Bench, workflows scored 75.8% versus 80.6% and ran in 1.3s instead of 170s.
- Three repair rounds lifted k=2 workflows from 66.7% to 81.2%.
- On eight SOP-Bench procedures, repair reached 87.8% at 0.40s per task.
- Without repair, SOP-Bench trailed few-shot generation, 80.6% versus 83.6%.
From the original article
1 min read at arxiv.orgBusiness process automation runs the same Standard Operating Procedure (SOP) again and again on new inputs. Large Language Model (LLM) agents can follow an SOP written in natural language, but they call an LLM to choose every step of every run, which adds latency and cost at scale.
By Kaiyuan Hou · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.