OT intrusion-detection scores can reflect recordings rather than attacks
Original title: Recording-Aware Evaluation of Multimodal OT Intrusion Detection
On ProvICS, same-recording tests reverse modality rankings and drop cyber-only fusion AUC to 0.54.
A paper argues that machine-learning OT intrusion detectors can look strong when benign and attack data come from separate recordings. The authors propose a recording-aware procedure that compares attack windows with same-recording background, fits models on prior data, tests temporal alignment, and reports event coverage with alarm burden. On ProvICS, cyber modalities already separate recordings before the first labeled attack, while same-recording evaluation favors the physical view. An external HAI analysis shows smaller, reference-dependent sensitivity.
Unconfirmed: Findings come from the authors' ProvICS study and a reference-dependent external HAI analysis.
- Cyber signals distinguish benign and attack recordings before labeled attacks.
- Physical data becomes the strongest modality under same-recording evaluation.
- Long alarm episodes can cover multiple actions and overstate detection.
- HAI shows smaller recording sensitivity than ProvICS.
From the original article
1 min read at arxiv.orgMachine-learning-based intrusion detection systems (IDSs) for operational technology (OT) are commonly evaluated on benchmark datasets in which normal and attack data are collected in separate recordings. In such settings, high performance may reflect recording-associated differences as well as attack behavior.
By Mustafa Umut Ozbek · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.