Knowledge That Is Linear Before Fine-Tuning Generalizes After It
How linearly readable a pretrained fact is strongly predicts whether fine-tuning on it generalizes.
Stories per day · 30 days
8 stories, newest first · one card per story
Stories tagged #representations
When VLMs Destroy What They Know: Spatial Representations Fall Below Lexical Priors
A new benchmark finds vision-language models drop spatial relations from the representations used downstream.
Predicting Alignment Generalization with Value Representations
Activation-based value representations predict LLM alignment generalization far better than textual descriptions of values.
Rep2Skill: Representation-Guided Skill Self-Evolution for LLM Agents
Rep2Skill revises LLM agent skills from internal representation trajectories and beats text-only self-evolution.
Do Audio Language Models Hear and Read Distinctive Features Alike?
Study finds most audio language models do not align heard and read features, except voicing in Qwen2.5-Omni.
Order-Invariant Answers, Order-Sensitive Representations in Mathematical Reasoning
Across 16 models from 1B to 8B, higher math accuracy tracks more distinct internal encodings of equivalent rule orders.
The Linear Representation Hypothesis Needs a Group Action
Researchers argue the Linear Representation Hypothesis is a family of claims distinguished by group-action equivalence.
Do speech foundation models really learn words?
Researchers show via residualization that later layers of HuBERT and wav2vec 2.0 encode word identity and semantics independently of phonetic content.