CoPE co-trains action forecasting with language-agent policies
Original title: Co-Evolving Acting and Forecasting for Language Agents
On ScienceWorld the authors report up to 1.9 times and 2.0 times the sample efficiency of ECHO and GRPO.
CoPE adds multi-step action targets so a shared policy backbone forecasts action sequences from successful on-policy trajectories while it learns to act. At inference the policy still takes one step at a time, without extra models or overhead. The authors report stronger aggregate results than GRPO and predictive-auxiliary RL baselines on ALFWorld, ScienceWorld, WebShop, τ^2-bench, and AppWorld. Code is published in the ulab-uiuc CoPE repository.
Unconfirmed: Efficiency and benchmark gains are author-reported and not independently verified.
- Forecast targets are taken only from successful on-policy trajectories.
- Each action is also supervised when predicted from earlier histories.
- Gains use one backbone, with no separate pretraining or inference overhead.
- Evaluation spans five simulated and application benchmarks.
From the original article
1 min read at huggingface.co
Training language agents is bottlenecked by the high cost of long-horizon rollouts, yet standard agentic reinforcement learning (RL) learns only from a terminal scalar reward, leaving the supervision in intermediate steps largely unused. Prior work exploits this supervision through value targets that estimate per-step success probabilities or observation targets that fit environment response.
By Haofei Yu, Yixing Chen, Yining Zhao · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.