JEV-as-a-Judge: Accept When Confident, Escalate When Unsure
A decision-only JEV judge nearly matches a top LLM judge at a fraction of the cost.
Stories per day · 30 days
3 stories, newest first · one card per story
Stories tagged #llm-as-a-judge
ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals
ImpossibleRubrics benchmark shows LLM-generated rubric reward signals are exploited 8-26% of the time by adversarial answers on impossible tasks.
When LLM judges agree, should we believe them?
Amazon ICML paper uses Ising models to correct correlated LLM-judge votes, beating accuracy-weighted panels by 9-14%.
2