Hybrid LLM judges achieve 4x lower cost for identity resolution
Original title: A Jev that can ask Opus for help gives Opus accuracy at 4x lower cost
Combining Jev with Claude Opus provides frontier accuracy at over 4 times lower cost than frontier models alone for grading identity resolution.
Researchers tested models for identity resolution, finding Jev and Luna perform well out of the box but a hybrid with Claude Opus reaches higher accuracy at lower cost. Decision models are 90x cheaper and faster but trail frontier accuracy by about 6 points. The hybrid improves maintenance and iteration for large-scale identity mapping.
- Hybrid Jev + Opus achieves frontier accuracy
- 4x lower cost than frontier models
- Luna better on easier cases
- Jev better calibrated on hard cases
- Decision models 90x cheaper than Opus
From the original article
11 min read at hightouch.com
We wanted to test whether LLMs and decision models, specifically Jev and Luna, could solve a hard problem that we encounter regularly (grading identity resolution results). We found that while both models performed okay out of the box (88. 8–97.
By theanomaly · hightouch.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.