Kierston simulation for crime investigation
Original title: Kierston & KierstonBench: Simulation of Allentown, PA and Evaluation of Language Models in Crime Investigation and Surveillance Tasks
KierstonBench benchmark tests language models on 125,000-resident Allentown simulation for record linkage and investigation tasks.
Kierston is a city-scale simulation engine modeling 125,000 residents of Allentown, Pennsylvania, generating daily activities and records. KierstonBench evaluates 32 task families across record linkage, routine reconstruction, incident investigation, and deceptive accounts. Most tested models fail to perform records investigation.
Unconfirmed: Simulation accuracy not independently verified.
- Models 125,000 residents of Allentown, PA
- Generates daily activities and records
- 32 task families in benchmark
- Most models cannot perform records investigation
- Frontier models solve most tasks and abstain when evidence insufficient
- Organizations
- Hugging Face
- AI models
- KierstonKierstonBench
From the original article
1 min read at huggingface.co
Large language models are candidates for collecting and analyzing records in criminal investigations and mass surveillance. Their capability and accuracy in this work cannot currently be measured. Real deployments lack the ground truth needed to score a model's conclusions, and no benchmark evaluates these capabilities in a simulated environment.
By Andrew Semenov, Svyatoslav Dorofeev · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.