LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders
Researchers use sparse autoencoders to localize trigger-based backdoor mechanisms in 1B and 8B LLMs, finding detection features differ from causal control features.
In a controlled language-switching backdoor setting where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German, the authors train sparse autoencoders (SAEs) across layers and transformer components. Attention and MLP features detect triggered prompts with near-perfect F1, but ablating them rarely suppresses the language switch, while residual-stream features can suppress triggered generation and some can induce target-language continuations without the trigger. The work decomposes token-trigger mechanisms into distinct SAE feature roles: trigger detection, residual-stream propagation, and language tracking, a decomposition the authors expect to transfer to other trigger-based backdoors.
- SAE features separate triggered prompts from translation and pretraining controls with near-perfect F1
- Trigger-detecting features do not necessarily causally control backdoor behavior
- Residual-stream feature ablation suppresses triggered language switching; some features induce it without triggers
- Experiments cover 1B and 8B language models with fixed trigger sequences
- Role-level decomposition may transfer to other trigger-based backdoors
From the original article
1 min read at arxiv.orgEven though backdoors in LLMs have been a growing concern, their inner workings are still under heavy scrutiny. Trigger-based backdoors are easy to define behaviorally, a rare input that makes the model switch to a chosen response pattern, but the mechanism between triggers and their responses is less clear.
By Wissam Antoun · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.