How It's Made: Uncovering Detection Engineering Processes for Network Intrusion Detection Rules
SuriCap study of 60 engineers writing 3,146 Suricata rules finds prior experience barely affects rule quality and identifies three rule-engineering phases.
Researchers introduced SuriCap, a platform for CTF-style network intrusion detection rule-engineering exercises, and ran workshops with 60 trained MSc students and experienced SOC professionals across four scenarios. The 3,146 valid rules produced showed prior experience had limited impact on rule quality, meaning less experienced engineers can match experts. Rules struggled to generalize beyond available test data, highlighting the need for sufficient labeled data. The study distills a three-phase rule-engineering pattern SOC managers can use to improve processes.
- SuriCap platform hosted CTF-style Suricata rule-engineering workshops
- 60 participants produced 3,146 valid rules across four scenarios
- Prior experience showed limited impact on rule quality
- Rules poorly generalized beyond available test data, requiring more labeled data
- Three-phase rule-engineering pattern identified for SOC process improvement
Full article129 words · extracted from arxiv.org · click to collapse
Many Security Operations Centers rely on signature-based Network Intrusion Detection Systems like Suricata, yet detection rule engineering remains understudied. We investigate this process by introducing SuriCap, a platform for rule engineering exercises, and hosting CTF-style workshops where 60 participants, trained MSc students, and experienced SOC professionals, created rules for four scenarios. Participants produced 3146 valid rules, enabling analysis of their methods, performance, and iteration patterns. Surprisingly, prior experience had limited impact on rule quality, suggesting that less experienced engineers can produce rules comparable to experts. We also observed challenges in generalizing rules beyond available tests, underscoring the need for sufficient labeled data. From our study, we identify three phases and a common pattern in rule engineering, offering SOC managers insights to improve their processes and expectations of engineer expertise.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.25901