Rethinking Backdoor Repair Evaluation: Distinguishing Aggregate Clean Utility from Benign Performance Preservation
Researchers show backdoor-repair scores can hide class-wise losses in benign model performance.
A cs.CR paper argues that backdoor-repair evaluations using Attack Success Rate and Overall Clean Accuracy can miss class-specific loss of benign performance. Aggregation can dilute localized loss or hide it through cross-class compensation. The authors propose Worst-Class Preservation Loss and Tail Preservation Loss and test them across attacks, repair methods, datasets, targets, and architectures, including clean-label attacks. They find substantial class-wise preservation losses can remain even when attack suppression looks successful.
- Overall Clean Accuracy can hide damage concentrated in a few classes.
- Class-wise preservation loss compares clean performance before and after repair.
- Worst-Class and Tail Preservation Loss capture localized degradation.
- Low attack success and high aggregate accuracy do not ensure uniform preservation.
- The pattern holds across attacks, repairs, datasets, architectures, and clean-label attacks.
Full article189 words · extracted from arxiv.org · click to collapse
Backdoor repair aims to suppress malicious behavior in compromised models while preserving benign task performance. Existing studies typically evaluate these objectives using Attack Success Rate (ASR) and Overall Clean Accuracy, but aggregate clean accuracy can obscure substantial degradation concentrated in a small portion of the label space. We revisit benign-performance evaluation from a preservation perspective by distinguishing aggregate clean utility from the preservation of previously available class-wise performance. We define class-wise preservation loss by comparing clean performance before and after repair and show that aggregation can hide localized degradation through localized-loss dilution and cross-class compensation. To complement Overall Clean Accuracy, we characterize localized preservation loss using Worst-Class Preservation Loss and Tail Preservation Loss. We conduct a systematic empirical study across representative backdoor attacks, repair methods, datasets, attack targets, and model architectures, with additional validation under clean-label attacks. Results show that effective attack suppression and favorable aggregate clean performance do not necessarily imply uniform preservation of previously available benign performance across classes. Substantial localized preservation losses can remain, and their severity and class-wise structure vary across repair conditions. These findings motivate preservation-oriented class-wise evaluation alongside ASR and Overall Clean Accuracy.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.25579