Diminishing Returns of Data in Sparse Autoencoders: Functional Behavior does not Guarantee Feature Reproducibility
SAE reconstruction converges on far less data, but one-to-one feature matches keep rising with more tokens.
The authors train four sparse autoencoder architectures on three language models with activation budgets from 5 million tokens to a 500 million-token reference. Reconstruction often approaches the full-data baseline at much smaller budgets, while one-to-one feature reproducibility continues to improve with more data. For Gemma-2-2B BatchTopK, strictly matched features rise from 2% at 5 million tokens to 29% at 50 million. Random sampling better preserves the activation distribution and reconstruction, whereas diversity-based sampling frequently improves AutoInterp.
- Four SAE architectures were trained on three models from 5M to 500M tokens.
- Reconstruction nears the full-data reference well before features become reproducible.
- Gemma-2-2B BatchTopK strict matches rose from 2% at 5M to 29% at 50M.
- Random sampling favored reconstruction; diversity sampling often improved AutoInterp.
- Tags
- sparse-autoencodersinterpretabilitygemma-2feature-reproducibilitydata-efficiencymechanistic-interpretability
- AI models
- Gemma-2-2B
From the original article
1 min read at huggingface.co
Sparse autoencoders (SAEs) are typically trained on hundreds of millions to billions of activation tokens, yet little work has tested whether this scale is necessary. We study how much activation data SAEs need by training four SAE architectures on three language models with budgets ranging from 5M tokens to a 500M-token reference.
By Sambit Ghosh, Madelyn Mathai, R. Venkatesh Babu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.