Essay claims neural nets are already largely interpretable

A Goodfire essay says most neurons already have one job and superposition was overstated, drawing a cautious response.

A Goodfire essay by Nick Cammarata argues that neural networks are already largely interpretable and that famous cases of polysemanticity were misread. He says superposition is the wrong explanation for InceptionV1, treating canonical examples as ordinary generalization and the “cat-car” neuron as a V-shape detector. Ordinary architectural changes, without altering model size or training data, reportedly made 99-100% of that model’s final-layer neurons interpretable across a 300-times size range, with smaller networks using coarser concepts and larger ones splitting them into specifics. A preliminary audit of Qwen3-32B estimates roughly 90% of neuron poles have one main job, about 7% have two related jobs, and 2-4% remain unexplained. Goodfire reportedly finds similar patterns in models up to trillions of parameters using AI-agent bulk analysis. Neel Nanda of Google DeepMind welcomes the evidence as a useful tool rather than a breakthrough, while Cammarata says readable networks would materially help AI safety. The two reports agree on the core claims and figures and do not contradict each other.

• A Goodfire essay by Nick Cammarata argues that apparent polysemanticity was largely misread and that superposition is the wrong, or less dominant, explanation for classic examples including InceptionV1.
• Cammarata says ordinary architectural improvements, without changing model size or training data, made 99-100% of InceptionV1’s final-layer neurons interpretable across a 300-times size range.
• He reinterprets canonical InceptionV1 superposition cases as ordinary generalization and the famous “cat-car” neuron as a V-shape detector.
• A preliminary Qwen3-32B audit estimates about 90% of neuron poles have one main job, about 7% have two related jobs, and 2-4% remain unexplained.
• Goodfire reportedly sees similar patterns in models up to trillions of parameters, with bulk analysis done by AI agents; smaller networks use coarser concepts and larger ones split them into specifics.
• Neel Nanda of Google DeepMind calls the evidence a useful tool but not an interpretability breakthrough; Cammarata says readable networks would materially help AI safety.