The Linear Representation Hypothesis Needs a Group Action
Researchers argue the Linear Representation Hypothesis is a family of claims distinguished by group-action equivalence.
The paper argues that representation claims which generalize beyond one trained model need an explicit equivalence between representations. It treats the Linear Representation Hypothesis as a family of claims, formalized with group actions that specify the representation object, the producing procedure, and the asserted property, including architecture-imposed equivalences. The framework is used to audit common representation quantities and recent interpretability analyses, showing that metrics, probes, and interventions can study different hypotheses.
- Treats the Linear Representation Hypothesis as equivalence-dependent claims
- Group actions specify the object, procedure, and asserted property
- Framework audits metrics, probes, and recent interpretability analyses
From the original article
1 min read at huggingface.co
To make claims about representations that generalize beyond a particular trained model, we need to specify when two representations should count as equivalent. The Linear Representation Hypothesis is often discussed without making this equivalence explicit.
By Louie Hong Yao, Yuhao Li, Shengchao Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.