The Geometry of Inference in Transformer Residual Streams
Six language models show residual streams become geometrically specific to their own final states during inference.
Stories per day · 30 days
3 stories, newest first · one card per story
Stories tagged #representation-geometry
Disentangling Representation Evolution in Transformers through Directional Decomposition
Decomposes transformer representation updates into parallel and perpendicular components, linking geometry to editing robustness and better pretraining.
Disentangling Representation Evolution in Transformers through Directional Decomposition
Researchers decompose transformer updates into parallel and perpendicular components, linking representation geometry to editing robustness, compression diagnosis, and training interventions.