Parts-of-Speech as Emergent Categories in SAE Latent Space
Sparse autoencoder latents encode part-of-speech in distributed groups, not one-to-one grammatical features.
The paper uses part-of-speech tags to test what linguistic structure sparse autoencoder latents expose in language models. PoS distinctions are highly recoverable from SAE activations but do not map one latent to one category, and the effect is not explained by lexical memorization. Open and closed classes differ, with categories supported by compact, overlapping groups of sparse latents that remain stable on held-out data. The authors conclude SAEs localize morpho-syntactic information in a distributed, category-dependent way rather than as atomic grammatical features.
- Part-of-speech is highly recoverable from SAE activations, not one latent per tag.
- The signal is not reducible to lexical memorization.
- Open and closed PoS classes differ substantially in feature support.
- Compact latent groups stay stable on held-out data but overlap related categories.
From the original article
1 min read at huggingface.co
Sparse AutoEncoders (SAEs) offer a promising way to inspect language model representations, but it is still unclear what kind of linguistic structure their latents expose. We use part-of-speech (PoS) categories as a controlled test case to study whether morpho-syntactic information is encoded by individual latents or by structured groups of features.
By Alessandro Bondielli, Lucia Passaro, Serena Auriemma · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.