User embeddings are superpositions of interpretable behavioral modes
Original title: User Embeddings are Superpositions of Interpretable Behavioral Modes
Sparse autoencoders recover named behavioral modes from embeddings and support zero-shot natural-language queries.
User embeddings compactly encode behavioral information but remain opaque. The authors argue they function as sparse superpositions of interpretable modes, a structure predicted by generative user-behavior models. Sparse dictionary learning recovers these modes from both classical and transformer embeddings. The resulting natural-language labels enable zero-shot audience retrieval without retraining.
- SAEs recover modes better than classical sparse baselines on synthetic data
- Method applies to transformer-encoded embeddings
- Enables zero-shot audience retrieval and characterization
- Post-hoc model-agnostic framework
- Automatically names and queries recovered modes in natural language
From the original article
1 min read at arxiv.orgUser embeddings are central components of modern recommendation, personalization, and behavioral modeling systems. Though these embeddings can compactly encode rich behavioral information, they are inherently dense and opaque, limiting interpretability. To address this gap, we argue that user embeddings can be understood as sparse superpositions of interpretable behavioral modes.
By Jonathan Kozaczuk · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.