RubricWorld improves LLM grading accuracy with world model-guided rubric optimization
Original title: RubricWorld: World Model-Guided Rubric Optimization for LLM Graders
Raises gpt-5-mini accuracy from 18.20% to 46.70% on ASAP2.0 exceeding baseline by 22.60 points.
RubricWorld learns from previous edits to predict correction and degradation rates. It selects candidates for evaluation using world model predictions. Outperforms baselines across multiple LLM graders on EIR, ASAP2.0, and ASAP-SAS datasets.
- Increases gpt-5-mini accuracy to 46.70% from 18.20% on ASAP2.0
- Highest score in 20 of 27 metric comparisons
- Uses paired grading results for training signal
- Code will be released upon acceptance
- AI models
- gpt-5-mini
From the original article
1 min read at arxiv.orgRubric optimization improves LLM grading by revising scoring instructions while keeping the grader's parameters fixed. Existing optimizers often judge edits by aggregate validation performance, limiting the supervision they retain about which errors an edit corrects or introduces. Testing each candidate also requires rescoring validation answers, which limits how many edits can be explored within a fixed budget.
By Hongji Pu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.