LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders
Researchers use sparse autoencoders to localize trigger-based backdoor mechanisms in 1B and 8B LLMs, finding detection features differ from causal control features.
In a controlled language-switching backdoor setting where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German, the authors train sparse autoencoders (SAEs) across layers and transformer components. Attention and MLP features detect triggered prompts with near-perfect F1, but ablating them rarely suppresses the language switch, while residual-stream features can suppress triggered generation and some can induce target-language continuations without the trigger. The work decomposes token-trigger mechanisms into distinct SAE feature roles: trigger detection, residual-stream propagation, and language tracking, a decomposition the authors expect to transfer to other trigger-based backdoors.
- SAE features separate triggered prompts from translation and pretraining controls with near-perfect F1
- Trigger-detecting features do not necessarily causally control backdoor behavior
- Residual-stream feature ablation suppresses triggered language switching; some features induce it without triggers
- Experiments cover 1B and 8B language models with fixed trigger sequences
- Role-level decomposition may transfer to other trigger-based backdoors
Full article225 words · extracted from arxiv.org · click to collapse
Even though backdoors in LLMs have been a growing concern, their inner workings are still under heavy scrutiny. Trigger-based backdoors are easy to define behaviorally, a rare input that makes the model switch to a chosen response pattern, but the mechanism between triggers and their responses is less clear. We study this mechanism in a controlled, harmless language-switching setting, where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German. For this, we train sparse autoencoders (SAEs) across layers and transformer components, then compare triggered prompts with translation and pretraining controls to identify trigger-relevant feature directions. We show how SAE features separate triggered prompts from controls with near-perfect F1, but features that detect the trigger do not necessarily control the behavior. In intervention tests, attention and MLP features often fire reliably on triggered prompts, making them good detectors, but ablating them rarely suppresses the language switch and activating them rarely induces it. In contrast, residual-stream features can suppress triggered generation when ablated, and some selected features can induce target-language continuations without the trigger. In short, these token-trigger mechanisms decompose into distinct SAE feature directions, with separate features for trigger detection, residual-stream propagation, and later language tracking. This role-level decomposition is the part most likely to transfer to other trigger-based backdoors, even when the payload, layers, or circuit locations differ.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.07746