Judging in Latent Space: Efficient Generative Reward Modeling via Semantics-Preserving Compression
LatentGRM judges preferences in compressed latent space, cutting 8B judge time about 6–7x while matching SFT accuracy.
LatentGRM is a latent generative reward model that uses semantic chunking, compression, and reconstruction so pairwise judgments can be made without generating textual assessments. Under matched training data and backbones, it reaches competitive aggregate preference accuracy versus explicit supervised fine-tuning judges at both 4B and 8B scales. Across four benchmark domains, LatentGRM-8B compresses evaluation trajectories by 8.9–9.2x and reduces total judge inference time by 6.1–7.0x at vote@5. A separate interpreter can reconstruct evaluation text, and controlled rubric interventions show criterion-dependent preference information is retained in the latent sequence.
- LatentGRM judges pairwise preferences from compressed continuous trajectories.
- At 4B and 8B it matches explicit SFT judges on aggregate preference accuracy.
- LatentGRM-8B compresses trajectories 8.9–9.2x and cuts inference time 6.1–7.0x at vote@5.
- Rubric interventions show criterion-dependent preference information survives compression.
- AI models
- LatentGRM-4BLatentGRM-8B
From the original article
1 min read at huggingface.co
Reward modeling often requires jointly representing and reasoning over multiple evaluation criteria, yet verbalizing this process token by token can incur substantial inference cost. Recent work on latent reasoning suggests that continuous states may support this computation more compactly. We introduce LatentGRM, a latent evaluation framework built on semantic chunking, compression, and reconstruction.
By Mingqing Yuan, Xiaobo Liang, Junwei Yang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.