H2R uses hierarchical hints as rewards for math reasoning
Original title: H$^2$R: Hierarchical Hint-Based Reward for Mathematical Reasoning via Reinforcement Learning
Qwen2.5-Math-7B rises from 30.4% to 39.1% on HiMath with hierarchical hints, and to 45.6% with oracle hints.
The paper proposes Hierarchical Hint-based Reward, which uses a math problem’s sub-question structure as intermediate signals during reinforcement learning with verifiable rewards. Hierarchical hints are treated as milestones that shorten the reasoning path to the final answer, unlike chain-of-thought prompting. The authors introduce HiMath, a benchmark of hierarchically structured problems. Across diverse LLMs, hints improved performance even without task-specific training, and H2R further raised Qwen2.5-Math-7B accuracy.
- Qwen2.5-Math-7B accuracy on HiMath rose from 30.4% to 39.1%.
- Oracle Hierarchical Hints reached 45.6% on the same model.
- HiMath is a new benchmark of hierarchically structured math problems.
- Hints improved reasoning across diverse LLMs without task-specific training.
From the original article
1 min read at arxiv.orgReinforcement Learning with Verifiable Rewards has shown great promise in enhancing mathematical reasoning in large language models. However, existing approaches provide limited intermediate supervision for complex, multi-step reasoning tasks.
By He Tu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.