LoRA subspace projection suppresses backdoors but does not erase them
Original title: Suppression Is Not Erasure: Mechanistic Analysis and Reactivation Boundaries of Subspace Projection for LoRA Backdoor Defense
Few-shot reactivation restored 91.7% to 100% attack success in one epoch for every weight-editing defense tested.
Researchers audit subspace-projection defenses for backdoored LoRA adapters across Qwen2.5 scales and InternLM2.5 and Spark. Rank-1 trigger-difference projection cut attack success from 60.4% to 0.0% on standard benchmarks and from 70.3% to 0.0% on a 1,000-sample suite in under 5 seconds. They find static success of 0.0% does not erase the backdoor: under 5-sample few-shot reactivation, evaluated weight-editing defenses rebound, all reaching 100.0% by epoch 2.
- Models tested were Qwen2.5-0.5B, 1.5B, 3B, and 7B, plus InternLM2.5 and Spark.
- Trigger-free calibration with benign polite suffixes reached 0.0% ASR without known triggers.
- Standard projections, Fine-Pruning, ANP, SVD truncation, and dual-end projection rebounded to 91.7%–100.0% ASR in one epoch.
- Their single-round dual variant reached 48.7% ASR in epoch 1 and 100.0% by epoch 2.
- First-epoch loss was up to 5.7 times lower than training from scratch.
From the original article
1 min read at arxiv.orgLow-Rank Adaptation (LoRA) is the standard paradigm for customizing large language models, yet third-party LoRA adapters pose a supply-chain risk: modifying less than 1% of model parameters suffices to implant stealthy backdoors. Recent defenses project adapter weights onto estimated backdoor subspaces, reporting near-zero static Attack Success Rates (ASR).
By Junsheng Fu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.