ControlScope: Workflow Revision and Reliability in LLM Agents
ControlScope finds only small, mixed gains from revising more of a running LLM-agent workflow.
ControlScope compares three repairs for a running LLM-agent workflow from the same public execution state: continue generated code (KEEP), edit the next tool call's arguments (ARG), or replace the unfinished workflow (FULL). On 20 filesystem tasks, FULL completed 15–16 tasks versus 13 for KEEP across reasoning-reviewer draws, while four fast draws completed 10–13 versus 13. ALFWorld fast panels scored KEEP/ARG/FULL at 85/86/87 on 87 tasks and 134/134/127 on 134 tasks; an AppWorld panel of 585 instances from 195 templates showed small net differences. Five-call protection saved 19.4% of logged model output and lost one success across 20 fresh source runs.
- Compares continuing code, editing arguments, or replacing the unfinished workflow.
- Filesystem FULL beats KEEP in some reviewer settings but not fast draws.
- ALFWorld and a 585-instance AppWorld panel show small net differences.
- Five-call protection saves 19.4% of logged output and loses one success.
From the original article
1 min read at huggingface.co
How much of a running workflow should a language model agent revise? ControlScope compares continuing generated code, editing the next tool call's data arguments, and replacing the unfinished workflow from the same public execution state. The nested permissions separate available repairs from the actions an agent selects.
By Jingjie Ning, Xueqi Li, Yibo Kong · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.