The Geometry of Inference in Transformer Residual Streams
Six language models show residual streams become geometrically specific to their own final states during inference.
The paper compares intermediate residual states with their own final states and endpoints from other contexts across six pretrained language models. Own endpoints become preferable to the average alternative early, while competitor sets generally shrink with depth and their membership changes. A high-dimensional model separates norm, alignment, and endpoint geometry, and a proof shows straight paths cannot introduce new competitors under Euclidean or cosine distance. Endpoints tied to lower-ranked output tokens tend to be farther away in cosine distance.
- Six pretrained language models compared via residual-to-endpoint geometry
- Competitor sets shrink with depth while membership still changes
- Straight paths cannot add new Euclidean or cosine competitors
- Lower-ranked token endpoints lie farther away in cosine distance
From the original article
1 min read at huggingface.co
Transformer language models build predictions through successive residual updates, but how their representations become specific to an eventual outcome remains unclear. We study this process by comparing intermediate residual states with their own final states and an empirical bank of final states from other contexts.
By Timur Mudarisov, Mikhail Burtsev, Radu State · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.