Calibration as a First-Class Criterion in LLM Evaluation
Authors argue LLM benchmarks should report calibration alongside accuracy for trustworthy evaluation.
The paper argues that NLP research often releases models, datasets, and benchmarks without checking whether confidence aligns with correctness. Miscalibration can harm deployment decisions and methods such as LLM-as-a-judge, synthetic data generation, and active learning. Because standard calibration metrics need only a confidence score and a correctness label, the authors say most current benchmarks can report calibration immediately, while open-ended generation remains unresolved.
- Argues every NLP metric should be paired with calibration.
- Overconfidence affects deployment and research methods.
- Most benchmarks already include confidence and correctness.
- Open-ended generation still lacks standard calibration inputs.
Full article180 words · extracted from huggingface.co · click to collapse
Calibration of language models -- the alignment between expressed or implicit confidence and empirical correctness -- is a well-studied subfield within NLP. Methods to measure it already exist. The problem is adoption: outside this subfield, NLP research regularly introduces new models, datasets, and benchmarks without checking whether the model's confidence scores are meaningful. We argue that this adoption gap is a major obstacle to trustworthy LLM evaluation. Miscalibration causes problems in two distinct areas: at deployment, where overconfident mistakes cause real harm, and inside the research pipeline, where methods like LLM-as-a-judge, synthetic data generation, and active learning rely on calibrated confidence without verifying it. Standard calibration metrics only require two inputs per example: a confidence score and a correctness judgment. Most benchmarks in use today already provide both, meaning calibration can be reported immediately. For open-ended generation, however, defining these two inputs is still an open challenge. We argue that each NLP subfield should pair its main performance metric with a calibration score and call for treating calibration as an essential property of every model rather than a niche topic.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.26489