What Did the Agent Actually Do? Evidence-Grounded Oversight for Long-Horizon Agents
AgentMonBench and an evidence graph help monitors verify consequential decisions made by long-horizon agents.
The paper studies oversight of long-horizon software-engineering agents, where the volume of activity makes it hard to decide which autonomous actions need human verification. AgentMonBench contains three subsets covering alignment between requirements and behavior and awareness of consequential decisions. The training-free Evidence-Grounded Behavior Graph groups source-linked evidence into behaviors and presents task-oriented views of their relationships. Experiments on eight models show improved decision identification and evidence localization versus raw context, with localization gains holding across input scales and hyperparameter settings.
- AgentMonBench covers alignment and consequential-decision awareness
- Training-free graph groups source-linked evidence into behaviors
- Eight models improve identification and evidence localization
- Localization gains persist across scales and hyperparameters
From the original article
1 min read at huggingface.co
As agents take on long-horizon tasks, users shift from making individual decisions to overseeing autonomous execution. Yet the volume of agent activity and the fragmentation of supporting evidence make it difficult to determine which decisions warrant user verification. We study monitors that identify consequential decisions and locate evidence to help users assess their implications.
By Zhongxiang Sun, Jiahao Yan, Hongkang Zhao · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.