How Much is a Human Right Worth? ECtHR-NPD: A Benchmark for Predicting Non-Pecuniary Damage Awardsnew
ECtHR-NPD benchmark covers 14,575 European Court of Human Rights cases for predicting non-pecuniary damage awards; LLMs struggle with zero and high awards.
Researchers introduce ECtHR-NPD, described as the first benchmark for predicting non-pecuniary damage awards at the European Court of Human Rights from case information where no statutory formula exists. It contains 14,575 cases with case-level awards in nominal euros, chronological splits, and a protocol separating target construction from model input. Evaluations covering constant predictors, gradient-boosted trees, retrieval methods, fine-tuned encoder LMs, prompted decoder LMs, and knowledge-augmented agents show sophisticated LM approaches do not consistently outperform the strongest feature-based baseline. All model families struggle to identify zero awards and to calibrate high-award predictions, with further degradation on a Challenging test view.