RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents
Researchers introduce RAFT, a stateful retrieval-augmented framework that retrieves timeline entries from historical cases to improve enterprise troubleshooting agents.
RAFT abstracts closed support cases into directed chains of timeline entries and retrieves at the entry level, returning parent-case trajectories anchored at matched states, with an optional case-level similarity graph. It beat vanilla RAG and GraphRAG baselines on Case Hit at every stage of case progress, using a synthetic benchmark built from Microsoft Learn Windows Server documentation and real Apache Jira issues with human-created duplicate labels. The benchmark, implementation, and Jira evaluation set are publicly released.
- Retrieves at timeline-entry level instead of whole documents
- Case-level graph links cases via configurable similarity
- Beats vanilla RAG and GraphRAG on Case Hit at every stage
- Evaluated on Windows Server docs benchmark and Apache Jira issues
- Benchmark, implementation, and Jira set released
Full article190 words · extracted from arxiv.org · click to collapse
Effective troubleshooting agents in enterprise customer support depend on retrieving actionable guidance from similar historical cases, yet existing retrieval-augmented generation (RAG) systems treat support cases as static documents and overlook their multi-stage, stateful nature. We introduce RAFT (Retrieval-Augmented Framework for Troubleshooting Agents), a stateful RAG framework that abstracts each closed historical case into a directed chain of timeline entries and retrieves at the entry level, surfacing cases whose intermediate states match the active case and returning the parent-case trajectory anchored at the matched state; an optional case-level graph links cases through a configurable similarity representation. We evaluate this retrieval layer directly, which, unlike evaluating a full agent system, requires no production deployment. Because public multi-stage troubleshooting data is extremely rare, we pair a synthetic benchmark built from Microsoft Learn Windows Server documentation with real Apache Jira issues carrying human-created duplicate labels. RAFT improves Case Hit over vanilla RAG and GraphRAG baselines at every stage of case progress, with statistically significant gains over the strongest baseline; the Jira results provide directional evidence that the advantage transfers to real case histories. We release our benchmark, implementation, and the Apache Jira evaluation set.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.20754