DeCert supervises generative reward models on rationale quality
Original title: Judging the Justification: Decision Certificate Supervision for Generative Reward Models
DeCert raises position-consistent accuracy by up to 14.8 points without reference explanations.
DeCert scores the validity and verdict consistency of natural-language rationales from generative reward models and turns those scores into preference supervision. When several samples share a correct verdict, certificate quality is used to prefer stronger justifications over weaker ones. Across two model families and in- and out-of-distribution benchmarks, the authors report better results than strong generative judge baselines and less positional bias.
Unconfirmed: Performance gains are author-reported and benchmark-specific; model names are not given.
- DeCert does not require reference explanations for rationale supervision.
- It builds preference pairs among rationales that share a correct verdict.
- Authors report gains on rationale quality and verdict accuracy.
- Position-consistent accuracy improved by up to 14.8 points.
From the original article
1 min read at huggingface.co
Generative reward models (GenRMs) generate natural language rationales to justify preference judgments, yet a correct verdict does not guarantee a sound justification. Existing approaches often rely on reference explanations to supervise rationales, but obtaining such references can be costly and does not necessarily establish whether generated rationales support the resulting verdict.
By Guiquan Sun, Bing Bai, Martin Renqiang Min · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.