Reasoning Topology Matters: A Controlled Study of LLM-Based Cybersecurity Analysis
A study finds graph-structured LLM reasoning lifts cybersecurity analysis accuracy by up to 12.2 points.
A cs.CR paper introduces Security Reasoning Topology, comparing Linear, Branching, and Graph organizations of intermediate LLM reasoning for cybersecurity analysis. Controlled tests cover MITRE ATT&CK network traffic, cyber threat intelligence, and CVE analysis, using Llama 2 at 7B, 13B, and 70B plus GPT-5.1 and Mistral Large 3 with fixed inputs. Graph reasoning was most accurate, improving 9.8 to 12.2 percentage points over few-shot prompting, while Branching was a strong middle option. The authors report the effect holds across model families and scales.
- Security Reasoning Topology compares Linear, Branching, and Graph structures.
- Evaluations used ATT&CK traffic, threat intelligence, and CVE analysis datasets.
- Models tested were Llama 2 7B, 13B, and 70B, GPT-5.1, and Mistral Large 3.
- Graph reasoning beat few-shot prompting by 9.8 to 12.2 percentage points.
- The topology effect stayed consistent across model families and scales.
Full article171 words · extracted from arxiv.org · click to collapse
Large Language Models (LLMs) are increasingly used in cybersecurity, where accurate analysis often requires multi-step and context-dependent reasoning over complex and heterogeneous data. However, existing prompting approaches typically focus on eliciting reasoning without explicitly considering how intermediate reasoning steps are structurally organized. We introduce Security Reasoning Topology, which models reasoning through three representative structures: Linear, Branching, and Graph. To evaluate their effects, we conduct controlled experiments on three cybersecurity datasets covering MITRE ATT&CK network traffic, cyber threat intelligence (CTI), and CVE vulnerability analysis. We evaluate multiple LLMs, including Llama 2 (7B, 13B, 70B), GPT-5.1, and Mistral Large 3, while keeping task inputs consistent and controlling reasoning structure through system-level prompting. Results show that reasoning topology substantially affects performance: Graph reasoning achieves the highest overall accuracy, improving over few-shot prompting by 9.8-12.2 percentage points across datasets, while Branching provides a strong intermediate solution. The results further show that the effect of reasoning topology remains consistent across model families and scales, highlighting reasoning topology as an important design factor for LLM-based cybersecurity analysis.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.24710