Outcome RL gains are mostly finishing and reliability, not reach
Original title: What Outcome RL Buys: Finishing, Reliability, Rarely Reach
On four physics and math sets, newly solved problems add at most 2.1 points, and only 3.5% of 144 unique physics solves held up.
The paper compares pass@1 for 20 released outcome-RL models, from 1.5B to 32B parameters, with the thinking models they started from, at fixed token budgets on physics and math. Gains split into finishing before the budget, being correct once finished, and reach on problems the base never solved in eight tries. Where bases run out of tokens, most of the change is finishing; with enough tokens, reliability dominates. Reach stays small, and a small 8B RL run showed finishing gains with flat reliability.
Unconfirmed: The 3.5% figure is the authors' check of 144 physics items, not an independent replication.
- Finishing accounts for 82% of the change at 4k and 8k token budgets when bases often hit the limit.
- Reach adds at most 2.1 percentage points on any of the four sets.
- Of 144 physics problems only the RL model solved, 3.5% held up on resampling or reading.
- In a human sample, nine of ten new RL steps also appeared in the base samples.
- Supplying each problem's physical setup did not beat plain RL in the reported check.
From the original article
1 min read at huggingface.co
Outcome RL raises pass@1 but not pass@k at large k (Yue et al. , 2025). But every pass@1 is scored under a token budget.
By Shan Yang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.