LexReward: A Taxonomy-Driven Reward Framework for Legal Language Models
LexReward defines taxonomy-driven rewards over Style, Element, and Chain dimensions for legal LLMs, feeding DPO and per-dimension reward models for RL.
LexReward characterizes legal response quality along three dimensions—Style (lexical/syntactic quality), Element (legal subjects, facts, statutes, decisions), and Chain (order, completeness, correctness, non-redundancy of reasoning)—with rubrics specifying criteria and quality levels per dimension. Rubric-based rewards construct pairwise preference data for Direct Preference Optimization and reward-model training, and DPO improves performance across all three dimensions. The resulting LexRM reward models support effective downstream reinforcement learning, each dimension-specific model improving its corresponding dimension without requiring reference answers at reward time.
- Three-dimension taxonomy: Style, Element, and Chain with explicit rubrics and quality levels
- Rubric rewards build DPO preference pairs; DPO improves all three dimensions
- Dimension-specific LexRM reward models drive RL gains without reference answers
From the original article
1 min read at huggingface.co
Legal language models require reward signals that capture not only answer correctness but also the multidimensional quality of legal responses. Existing reward methods, however, often rely on coarse-grained holistic judgments, providing limited domain specificity and interpretability. We introduce LexReward, a taxonomy-driven framework for legal reward modeling.
By Yida Cai, Xin Dai, Bingxiang He · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.