KnowGPO: Knowledge-Grouped Policy Optimization for Search Agents
KnowGPO credits search-agent steps by knowledge gained, lifting an 8B model 3.8–4.2 points over process-RL baselines.
KnowGPO is a reinforcement-learning method for deep search agents that credits steps by the task-relevant knowledge acquired rather than surface trajectory states. For each query it decomposes the answer into atomic knowledge conditions and groups trajectories that share the same satisfied subset. On Xbench, BrowseComp-ZH, and GAIA it beats outcome-only GRPO and process baselines GiGPO and IGPO by 3.8 to 4.2 points, with an 8B model rivaling most 32B open-source systems.
- KnowGPO groups trajectories by satisfied atomic knowledge conditions rather than surface observations.
- Incremental knowledge gain is used as an environment-grounded process reward.
- An 8B model gains 3.8–4.2 points over GRPO, GiGPO, and IGPO on Xbench, BrowseComp-ZH, and GAIA.
- The 8B system rivals or exceeds most 32B open-source search agents.
From the original article
1 min read at arxiv.orgDeep search agents solve knowledge-intensive tasks through long-horizon, multi-turn interactions, where assigning credit to individual search decisions is fundamentally difficult. Outcome-based reinforcement learning provides only a single terminal reward shared across an entire trajectory.
By Siqi Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.