Semifactual Credit-Augmented Policy Optimization
SCAPO uses semifactual token stability to beat GRPO on AIME for two Qwen3 base models.
The paper studies how verifiable-reward reasoning models respond to semifactual prompt changes that preserve the problem and its answer. It finds token-level probability drift, and that suppressing high-drift candidates during decoding can improve accuracy without weight updates, exposing a weakness of GRPO's uniform token advantage. SCAPO uses normalized semifactual stability to reduce advantages for unstable tokens early in training without granting extra credit for stability alone. On Qwen3-4B-Base and Qwen3-1.7B-Base, it improves AIME 2024-2026 accuracy over GRPO by 5.63 and 4.17 percentage points and leads most math and all evaluated out-of-distribution benchmarks.
- Semifactual edits reveal token sensitivity to irrelevant prompt features.
- SCAPO reduces early-training credit for unstable response tokens.
- AIME accuracy rises 5.63 and 4.17 points over GRPO.
- Leads most math and all tested out-of-distribution benchmarks.
From the original article
1 min read at arxiv.orgReinforcement learning with verifiable rewards (RLVR) has improved the reasoning capabilities of large language models (LLMs), yet their predictions remain sensitive to task-irrelevant prompt features. We investigate this sensitivity through semifactual prompt interventions that preserve the underlying problem and its answer.
By Junshu Pan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.