Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks
Researchers propose Feedback-Enriched Environments (FEEs) that reduce reward sparsity and improve RL training of Qwen3-based agents on SciWorld and BFCL.
The paper proposes shifting from agent-side warmup (SFT) to environment-side adaptation via Feedback-Enriched Environments to address severe reward sparsity in RL training of long-horizon LLM agents. A pilot study defines a feedback strategy that transitions from action guidance to observation enrichment in later training stages. Large-scale experiments on SciWorld and BFCL across Qwen3 model scales and GRPO, GSPO, and DAPO show consistent gains, plus stabilized training dynamics and proactive exploration.
25