Self-Credit densifies sparse rewards for long-horizon agentic RL
Original title: Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling
It raises Qwen3 8B and 32B by 37.0 and 36.2 points on BrowseComp-Plus over the base models.
Self-Credit is an agentic reinforcement-learning method that turns sparse outcome rewards into denser credit by having the policy cite earlier turns that contain critical evidence. Intra-group rescaling compares those citations across successful and failed rollouts for the same prompt, and advantage reweighting updates the cited turns. On BrowseComp-Plus it also beats the strongest reported agentic RL baselines by 8.5 and 5.6 points on Qwen3 8B and 32B.
- Gains over the strongest baselines are 4.9 points on GAIA and 4.7 on xBench-DeepSearch.
- Rescaling uses agreement and disagreement inside each prompt's rollout group.
- Analyses say rescaling on failed rollouts matters across model scales.
- The policy writes summaries with citations during rollouts.
From the original article
1 min read at huggingface.co
LLM agents are commonly trained with RLVR, but the sparse nature of typical RLVR rewards makes credit assignment challenging, especially for long-horizon tasks. We propose Self-Credit, an end-to-end agentic RL method that creates dense supervision signals from sparse outcome rewards using self-generated credit assignment via intra-group rescaling. Self-Credit builds on group-based RL with three components.
By Fangcong Yin, Ming Yin, Tanvi Bhandarkar · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.