Smart-grid detector study finds shuffled tests overstate attack detection
Original title: Interpretable Cyber-Physical Attack Detection in Smart Grids: A Leakage-Controlled Multi-Testbed Evaluation of Fusion, Robustness, and Observability Limits
Temporal evaluation drops macro-F1 from 0.96 to 0.88, and stealthy false-data injection defeats all three observation views.
A multi-testbed study evaluates an interpretable, time-windowed fusion Random Forest for cyber-physical smart-grid attack detection and reports the more pessimistic result when protocols disagree. Shuffled cross-validation is optimistic, and feature-space attacks that are not realistic grid manipulations cut macro-F1 to 0.67 and 0.34. On real IEC-104 traffic, structural attacks are caught far more often than semantic value manipulation, while stealthy false-data injection defeats all observation views. Code and benchmarks are released.
Unconfirmed: Feature-space attacks are described as infeasible as realistic grid manipulations.
- Macro-F1 falls from 0.96 to 0.88 ± 0.06 under rolling-origin evaluation.
- Adaptive and adaptive-joint attacks reduce macro-F1 to 0.67 and 0.34.
- IEC-104 structural-attack recall is 0.87 to 0.98; semantic manipulation recall is 0.11 to 0.35.
- On IEEE 118-Bus, the manifold test detects every trial down to 0.05 degree/step.
From the original article
1 min read at arxiv.orgCyber-physical smart-grid detectors are typically treated as black boxes and validated exclusively on clean, temporally shuffled datasets, which risk overly optimistic performance claims. This work presents a multi-testbed evaluation study built around an interpretable, time-windowed fusion Random Forest, adopting the pessimistic result whenever evaluation protocols disagree.
By Muhammad Ridwanul Hoquea · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.