60
60
30
45
55
30
55
30
60
42
30
60
42
55
60
42
45
30
30
60
PIA-Bench: Towards Automated Privacy Impact Assessment with Large Language Models
Researchers release PIA-Bench, the first open benchmark evaluating how accurately LLMs can automate privacy impact assessments using 73 curated federal PIAs.
PIA-Bench is the first open benchmark for evaluating large language models on real-world privacy impact assessments (PIAs). The authors audited 499 expert-authored PIAs published by US federal agencies and curated 73 structured PIAs comprising 451 privacy risk items and 831 mitigation items. Off-the-shelf LLMs were found to produce meaningful assessments while identifying clear avenues for improvement. The paper calls for domain-specific LLM agent workflows, accountable LLM infrastructure, and new quality standards for PIAs.
48
30
55
42
30
60
42
30
60
57
30
60
57
60
30
60
30
60
60
60