Can Generative AI Automate Data Extraction for Meta-Analysis? A Case Study on Intercropping Research
A study finds zero-shot LLM prompting extracts intercropping meta-analysis data better than staged or multi-agent workflows, but remains inaccurate.
Researchers compared direct zero-shot prompting, a staged workflow, and a multi-agent system across six open-weight models for extracting data from intercropping literature. Outputs were scored against a manually curated ground truth and checked with a downstream statistical analysis. Direct zero-shot prompting was strongest, with a mean similarity-adjusted F1 of 0.577, but no method approached full accuracy. Most model-approach pairs recovered the direction of predictor-outcome relationships without estimating magnitude accurately.
- Compared zero-shot, staged, and multi-agent extraction with six open-weight models.
- Zero-shot prompting led with mean similarity-adjusted F1 of 0.577.
- No approach was close to fully accurate extraction.
- Downstream analysis usually recovered relationship direction, not magnitude.
From the original article
1 min read at arxiv.orgMeta-analysis is the synthesis of information from multiple sources to arrive at an overarching conclusion. There is a large need for meta-analysis in agricultural research to synthesize what is known and analyze overarching patterns.
By Zehao Lu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.