Study tests whether causal neuron groups are unique or transferable
Original title: Are Causal Neuron Circuits Unique and Transferable?
A few neurons often flip two-hop answers, but effects weaken on other chains.
The paper asks how much of a neuron group is necessary when ablating or patching it changes a language model’s answer. On a capital task in Qwen3-0.6B, zero-ablating 12 selected MLP neurons lowers the answer margin by 27%, while random neurons barely change it. On two-hop chains, patching 12 neurons often shifts the answer to the bridge entity across Qwen, Llama and Gemma models. Full groups are rarely necessary, and effects transfer to rephrasings but are weaker on other chains.
- Authors measure all 4,096 subsets of each selected group of 12 neurons.
- Bridge-entity flips occur in 9/11 Qwen3-1.7B and Llama-3.2-1B chains.
- Gemma-3-1B flips in 8/10 chains; random neurons never do.
- Disjoint subsets can each flip an answer, often when confidence is lower.
- The study does not recover full circuit connectivity.
From the original article
1 min read at huggingface.co
Interpretability demonstrations often show that intervening on a group of neurons changes a model's answer. A recent example removes an 18-neuron group in Qwen3-32B and the model answers "Simba" instead of "Scar" to a two-hop Lion King riddle.
By Ömer Tekin Yavuz · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.