CAV verifies LLM investigator evidence against tool conflicts
Original title: When AI Investigators Cannot Trust Their Tools: Conflict-Aware Evidence Verification for Cyber Threat Investigation
CAV-LLM with vote gate and nearest-neighbor check maintains 0.84-0.88 accuracy under heavy tool corruption versus naive investigator dropping to 0.08.
LLM agents triage alerts by combining detection tool outputs assuming trustworthiness. Conflicts arise among tools themselves under corruption. On corrected CIC-IDS2017 dataset heterogeneous detectors emit labels and confidences. CAV uses equal-weight vote evidence-conflict gate and independent nearest-neighbor trust check. It improves accuracy across corruption levels calling LLM on only 0.4% clean traffic.
- CAV-LLM reaches .84-.88 with three corrupted detectors
- Naive accuracy falls to .13 or less
- Honest disagreement causes up to 63% abstention
From the original article
1 min read at arxiv.orgLLM agents increasingly triage security alerts by combining the outputs of several detection tools, implicitly treating those tools as trustworthy. Unlike prior work on conflicts between a model's knowledge and its context, we study conflicts among the tools themselves: what happens when this machine-generated evidence is conflicting or wrong.
By Yuchen Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.