Rubric Rewards from Item Response Theory
Rubric Response Theory turns rubric verdicts into scalar RL rewards and cuts judge calls while beating GRPO.
Rubric Response Theory models monotone rubric criteria with a two-parameter item response model, turning verdict patterns into a scalar quality score instead of summed points. A Response Parameter Network predicts criterion difficulty and discrimination from the prompt and criterion text and is updated online by expectation maximization. With Qwen3.5-4B, RRT's macro criterion score is 1.7 points above GRPO across Medical, Science, Rubrics as Rewards Science, and RubricBench, with gains of 2.8 to 5.6 points on hard criteria. Adaptive Fisher selection at half the criterion budget stays within 0.1 points of full-judging GRPO.
- RRT scores rubric verdict patterns with a two-parameter item response model.
- A Response Parameter Network predicts difficulty and discrimination and updates online.
- Qwen3.5-4B gains 1.7 macro points over GRPO across four datasets.
- Hard Medical and Science criteria improve by 2.8 to 5.6 points.
- Half-budget Fisher selection stays within 0.1 of full GRPO judging.
From the original article
1 min read at huggingface.coMany language tasks have no single answer that can be checked automatically. Rubrics provide criteria for judging responses to these tasks. For reinforcement learning, the resulting verdicts must be combined into a scalar reward.
By Milad Yazdani, Yaser Souri, Xiren Zhou · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.