Dissecting Agentic Forensics: The Role of Triage, Prompting, and Evidence Arbitration in Open-World Fake Image Detection
Agentic fake-image forensics depends more on the MLLM judge's reasoning than on detector fusion.
A cs.CR paper evaluates a training-free agentic image-forensics framework that combines specialist detectors, per-detector triage, and conflict-aware evidence arbitration. Across six configurations and three multimodal LLM backbones, naive detector fusion produced severe false positives on authentic images, while triage and prompting improved calibration. Reasoning quality dominated: a stronger judge beat a weaker one, especially out of distribution. Manipulation recall was nearly saturated, so the main open-world challenge is trusting and arbitrating forensic evidence rather than finding edits.
- Naive fusion of specialist detectors produces severe false positives on authentic images.
- Triage and prompting filter unreliable evidence and expose detector limitations.
- A stronger multimodal judge outperforms a weaker one, especially under distribution shift.
- Manipulation recall is nearly saturated; the hard problem is calibrating trust.
Full article204 words · extracted from arxiv.org · click to collapse
Image forensics is increasingly an open-world problem: manipulations range from fully synthetic images to localized edits, splicing and swapping, while most forensic detectors remain specialized to a single manipulation family. Agentic AI has recently emerged as a promising solution. In principle, such systems can assess the reliability of individual detectors, identify out-of-scope evidence, and arbitrate conflicting reports. However, it remains unclear which components actually drive performance and whether their benefits persist under distribution shift. To answer these questions, we study a training-free agentic framework built around specialist detectors, per-detector triage, and conflict-aware evidence arbitration. Using six configurations and three multimodal large language model backbones, we dissect the role of triage, prompting, and reasoning quality on both in-distribution and out-of-distribution data. Our results show that naive detector fusion suffers from severe false-positive rates on authentic images. Triage and prompting consistently improve performance by filtering unreliable evidence and exposing detector limitations. However, the dominant factor is represented by reasoning itself: A stronger judge substantially outperforms a weaker one, particularly under distribution shift. Most notably, manipulation recall is nearly saturated across all configurations, indicating that the main challenge of open-world image forensics is not detecting manipulations, but calibrating trust in specialized forensic tools and arbitrating conflicting evidence.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.24359