Learning to Steer LLM Agents and Multi-Agent Systems
Researchers train a reinforcement-learning steering agent that detects goal drift and corrects LLM agents during execution.
LLM agents and multi-agent systems can drift from their goals during long-horizon tool use, wasting compute and producing bad outcomes. The authors propose a steering agent that monitors execution and chooses either a cheap fast intervention or a costly slow one. Both paths are trained jointly with reinforcement learning using pairwise ranking and group-relative policy optimization (GRPO). On information-seeking search tasks, the method improves single-agent and multi-agent performance and efficiency, and it generalizes to new tasks and unseen executors.
- A steering agent intervenes when LLM agents drift from the task goal.
- Fast and slow paths are trained together with ranking and GRPO.
- Supervision improved search-task performance and execution efficiency.
- The policy transfers to unseen tasks and executors without retraining.
From the original article
1 min read at arxiv.orgLarge language model (LLM)-based agents and multi-agent systems increasingly tackle complex tasks through long-horizon tool use and inter-agent collaboration. However, these systems are prone to deviating from the overall goal as execution unfolds, which wastes execution resources and leads to undesired outcomes, a failure termed goal drift.
By Leyang Shen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.