RL on expert tasks lifts NVIDIA Nemotron 3.5 Super on 12 benchmarks
Original title: What RL on Expert-Built Tasks Teaches Nemotron 3.5 Super
Five separate rank-32 LoRA adapters, each trained with CISPO and no supervised stage, are compared with the base model.
NVIDIA post-trains Nemotron 3.5 Super (120B total, 12B active) with reinforcement learning on five expert-built datasets, one at a time: software engineering, complex constraints, long-horizon tool use, chart understanding, and enterprise document reasoning. Each run trains a fresh rank-32 LoRA with CISPO on Tinker, without a supervised stage. On twelve headline benchmarks, reported gains range from +4.5 on IFBench to +13.8 on GDP_pdf.
Unconfirmed: Benchmark gains are reported by the authors and not independently verified.
- GDP_pdf rises from 10.4% to 24.2%; SWE-bench Pro V2 from 86.9% to 92.1%.
- Terminal-Bench 2.1 rises from 63.9% to 70.3%; MultiChallenge from 58.3% to 66.6%.
- AutomationBench rises from 14.8% to 27.5%; Chartography from 39.4% to 50.6%.
- The document adapter, trained without API tools, gains 8.0 points on AutomationBench.
- Eight benchmarks were built by other groups and four by Surge AI.
- Vendors
- NVIDIA
- Organizations
- Surge AI
- AI models
- Nemotron 3.5 Super
From the original article
1 min read at arxiv.orgWe post-train NVIDIA's Nemotron 3. 5 Super (120B total, 12B active parameters) with reinforcement learning (RL) on five expert-built task datasets, one at a time: real-world software engineering, complex constraints, long-horizon tool use, professional chart understanding and enterprise document reasoning.
By Sushant Mehta · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.