Bridging the First-Hour Gap: Evaluating AI Reliability and Benchmarking Deficiencies in Cyber Incident Response for Law Enforcement
Survey of playbooks, LLMs, RAG, and agentic AI for law-enforcement cyber first responders finds RAG most viable but benchmarks inadequate for legal requirements.
The paper surveys decision-support architectures (playbooks, LLMs, RAG frameworks, agentic AI) for frontline law enforcement during the first hour of a cyber incident, where volatile digital artifacts risk procedural errors and evidence attrition. RAG-based systems are identified as a relatively viable intermediate solution, though prompt sensitivity and confident hallucinations in legal contexts pose major risks. The authors find current cybersecurity benchmarks insufficient for law enforcement safety and legal demands, and argue for a new benchmark focused on naive query robustness and evidence preservation.