Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures
Continual Search framework iteratively prompts LLM judges to keep searching agent execution logs, boosting long-horizon failure root-cause attribution accuracy.
The paper frames automated root-cause attribution (RCA) for long-horizon AI agent failures as a search problem, since relevant evidence is sparse and distributed across massive execution traces. The authors propose Continual Search, an iterative framework that nudges an LLM judge across successive turns to keep hunting unresolved diagnostic evidence instead of settling on an early plausible diagnosis. They introduce MegaRCA-Mix, a benchmark of 50 human-annotated failure trials on long-horizon, execution-heavy tasks. On MegaRCA-Mix, Continual Search improves GPT-5.5's F1 from 0.349 to 0.498 (over 40% gain), and lower-tier models can surpass higher-tier counterparts when search is effective.
- Automated RCA for agent failures framed as a massive search problem
- Continual Search iterates LLM judging across turns instead of one-shot diagnosis
- New MegaRCA-Mix benchmark: 50 human-annotated long-horizon failure trials
- GPT-5.5 F1 on MegaRCA-Mix rises from 0.349 to 0.498
- Effective search can beat raw model scale within a model family
From the original article
1 min read at huggingface.co
The increasing deployment of AI agents in long-horizon tasks yields massive execution logs. Diagnosing failures within these records is crucial for reliability, as it transforms outcome-level signals into actionable interventions. The sheer scale of the data renders human review impractical, driving the need for automated root-cause attribution (RCA).
By Harsh Raj, David Lee, Anas Mahmoud · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.