Learning to Read Out: Unembedding Dynamics in Language Model Pretraining
Pythia encodes task information in hidden states before its unembedding readout can express it.
Stories per day · 30 days
Relatedcontrastive-learning1crosscoding1embeddings1evidence-alignment1interpretability1language-models1mmeb1multimodal1pretraining1pythia1retrieval1unembedding1
2 stories, newest first · one card per story
Stories tagged #readout
Learning Multimodal Embeddings with Evidence-Aligned Readout
EviAlign reads multimodal evidence at semantic boundaries, reaching 76.9 Recall@1 across 12 MMEB tasks.