Goodfire researcher claims neural nets are more interpretable than thought
Original title: “We were looking at it wrong”: The explosive claim that’s upended the AI world
Stanford's Chris Potts calls the 26,000-word post a hunch that does not make interpretability easier.
MIT Technology Review reports that Nick Cammarata of Goodfire published a 26,000-word post arguing that large language models may be easier to understand, monitor, and control than researchers assumed. He revisits superposition, including an Inception v1 neuron previously linked to both cars and cats, and says closer inspection revealed shared structure such as a V shape. The piece notes wide online reaction after reports of agent containment failures, while Stanford's Chris Potts says the post is little more than a hunch.
Unconfirmed: Cammarata's claim that networks are far more legible is disputed and is not presented as a verified result.
- Cammarata previously worked on mechanistic interpretability at OpenAI.
- He says he spent most of the last year inspecting neurons at Goodfire.
- He claims new tools and AI agents helped map circuitry in Inception v1.
- Potts says prominent social-media reaction overstates the post as a breakthrough.
- Cammarata says he did not expect readers to update their beliefs so strongly.
- Products
- Inception v1
- Organizations
- GoodfireOpenAIStanford University
From the original article
8 min read at technologyreview.com
“I think I have good news about the nature of neural networks.” With those simple words of optimism, Nick Cammarata, a researcher at the AI startup Goodfire, kicked off a 26,000-word blog post that has knocked the AI world on its head, sparking wild speculation as well as cautious (and not-so-cautious) celebration. Cammarata’s claim is…
By Will Douglas Heaven · technologyreview.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.