The Linear Representation Hypothesis Needs a Group Action
Researchers argue the Linear Representation Hypothesis is a family of claims distinguished by group-action equivalence.
The paper argues that representation claims which generalize beyond one trained model need an explicit equivalence between representations. It treats the Linear Representation Hypothesis as a family of claims, formalized with group actions that specify the representation object, the producing procedure, and the asserted property, including architecture-imposed equivalences. The framework is used to audit common representation quantities and recent interpretability analyses, showing that metrics, probes, and interventions can study different hypotheses.
- Treats the Linear Representation Hypothesis as equivalence-dependent claims
- Group actions specify the object, procedure, and asserted property
- Framework audits metrics, probes, and recent interpretability analyses
Full article139 words · extracted from huggingface.co · click to collapse
To make claims about representations that generalize beyond a particular trained model, we need to specify when two representations should count as equivalent. The Linear Representation Hypothesis is often discussed without making this equivalence explicit. Different notions of equivalence preserve different structures, so metrics, probes, and interventions that appear to study the same representation may in fact correspond to different hypotheses. We therefore argue that the Linear Representation Hypothesis is not one hypothesis but a family of claims distinguished by representation equivalence. We formalize this idea using group actions, specifying the representation object, the procedure that produces it, and the property ultimately asserted, while accounting for equivalences imposed by the model architecture. This framework clarifies how assumptions can change across metrics, reading points, and analysis stages, and we use it to audit common representation quantities and recent interpretability analyses.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.27158