Evidence-Grounded Auditing of Identification Assumptions in Climate-Policy Causal Evaluations
ARGUS, a language-model pipeline, audits identification assumptions in climate-policy difference-in-differences studies and abstains when evidence is missing.
Researchers introduce ARGUS, a language-model pipeline that audits difference-in-differences climate-policy studies against an eleven-dimension assumption-implication-evidence rubric and abstains when evidence cannot be retrieved. On an 11-flaw benchmark it detects 73% of planted flaws, compared with 18% for a keyword pipeline. Across 26 economics papers it abstains on about 40% of paper-dimension assessments. In a five-paper pilot it assigns higher risk than reconciled labels on 25 of 33 completed assessments, and the authors present it as a review aid rather than a judge of causal claims.
- Detects 73% of planted flaws versus 18% for a keyword pipeline
- Abstains on about 40% of assessments across 26 economics papers
- Five-paper pilot rates higher risk than labels on 25 of 33 assessments
- Localizes evidence-linked weaknesses for experts without adjudicating causal claims
From the original article
1 min read at huggingface.co
Difference-in-differences (DID) studies are widely used to evaluate climate policy, but assessing the evidence supporting their identification assumptions remains challenging. We introduce ARGUS, a structured language-model pipeline that audits reported evidence against an eleven-dimension assumption-implication-evidence rubric and abstains when relevant evidence cannot be retrieved.
By Yonghong Zhang, Yong Xie, Isabel M. Parra · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.