Behavioral Compositionality of Sparse Autoencoder Representations
SAE features are more compositional than random bases, but faithful explanations still need pairwise interactions.
The paper introduces SAE Additivity Residuals (SAR) to measure behavioral effects of sparse autoencoder features that do not compose additively. Across 18 model and behavior settings, learned SAE coordinates show lower first-order residuals than sum-preserving random rotations of the same subspace. Pairwise terms improve held-out coalition ordering everywhere. In a combined steering experiment, pairwise modeling raises rank fidelity from 0.5874 to 0.7790 and reduces ordering error from 0.1444 to 0.0606.
- SAR measures behavioral variation left unexplained above a chosen interaction order.
- Learned SAE coordinates beat random rotations on first-order residuals in all 18 settings.
- Pairwise terms improve held-out coalition ordering in every setting.
- Pairwise steering raises rank fidelity from 0.5874 to 0.7790.
From the original article
1 min read at huggingface.co
Sparse autoencoders (SAEs) represent model activations with sparse, linearly decoded features, but the downstream effects of those features need not compose additively. We introduce SAE Additivity Residuals (SAR), which measure the behavioral variation left unexplained above a chosen interaction order.
By Haotian Ma, Ziye Mei, Frederic Sala · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.