Understanding and Enhancing Backdoor Persistency in LLM Agent Post-Training
Study finds coding-agent backdoors survive benign SFT and RL, and PersistBD greatly increases their persistence.
The paper examines a supply-chain risk in which a backdoored third-party model is adapted into a software-engineering agent through benign supervised fine-tuning and task-level reinforcement learning. Benign SFT substantially reduces attack success, but subsequent RL often preserves the residual behavior and can increase it. The authors' PersistBD method refines an already-backdoored model before release. On Qwen2.5-Coder-7B it raises attack success from 20% to 74% after SFT and from 20% to 76% after SFT plus RL, while benign task performance stays comparable.
- Benign SFT lowers backdoor success, but later RL can preserve it
- Persistence linked to initial strength and gradient compatibility
- PersistBD raises post-SFT success from 20% to 74% on Qwen2.5-Coder-7B
- Post-SFT-RL success rises from 20% to 76% with similar benign performance
Full article201 words · extracted from huggingface.co · click to collapse
Developers can build LLM agents by adapting third-party models through benign post-training. We study a supply-chain threat in which an attacker supplies a model with a backdoor: hidden behavior that produces malicious outputs when a particular input pattern appears. Focusing on software-engineering agents, we ask whether such backdoors survive the developer's supervised fine-tuning (SFT) and subsequent task-level reinforcement learning (RL). We observe that benign SFT substantially reduces attack success, but subsequent RL often preserves the residual behavior and sometimes even increases attack success. Our analysis of backdoor erosion during SFT identifies two factors that may favor survival: initial backdoor strength and gradient compatibility with benign training. These factors motivate PersistBD, which refines an already-backdoored model before release to improve its persistency through the benign post-training process. On Qwen2.5-Coder-7B, PersistBD raises attack success from 20% to 74% after SFT and from 20% to 76% after SFT-RL, while maintaining comparable benign task performance. Together, our results show that backdoors can remain active through benign post-training and that adversaries can deliberately increase their persistence. This highlights a supply-chain risk for AI developers and motivates stronger techniques for detecting and mitigating inherited backdoors when adapting third-party models into agents. Our code is available at https://github.com/uiuc-kang-lab/PersistBD.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2610.07510