Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs
Study shows LLM reasoning operations like planning and deduction are geometrically separable in hidden states, with separability peaking in middle layers.
Researchers investigate whether functional reasoning operations — problem formulation, goal decomposition, deduction — have corresponding geometric structure in LLM hidden representations. They find operations are separable in held-out representations with separability peaking in middle layers, ruling out lexical and positional confounds; token-wise operation alignment becomes more distributed across layers, and identical surface tokens are represented differently depending on their surrounding chunk. Attention-masking interventions show chunk-onset operation-aligned representations depend on preceding reasoning context; code is released on GitHub (naver-ai/beneath-cot).
- Reasoning operations are separable in hidden representations
- Separability peaks in middle layers
- Identical tokens represented differently by surrounding operation chunk
- Attention masking shows dependence on prior reasoning context
Full article148 words · extracted from huggingface.co · click to collapse
Reasoning in large language models unfolds through diverse functional operations, such as problem formulation, goal decomposition, and deduction. Although these operations are explicitly distinguished in text, little is known about how they are geometrically organized in representation spaces. To this end, we investigate whether distinct reasoning operations exhibit corresponding geometric structure in hidden representations. We find that operations are separable in held-out representations, with separability peaking in middle layers, and verify that this structure is not explained by lexical or positional confounds. Across layers, token-wise operation-alignment becomes more distributed over spans, while identical surface tokens are represented differently depending on the operation of its surrounding chunk. Attention-masking interventions further show that operation-aligned representations at chunk onset depend on preceding reasoning context. Consequently, our work demonstrates that language models maintain representational correspondence between linguistic reasoning expressions and their internal geometric structures. Code and project materials are available at https://github.com/naver-ai/beneath-cot.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.04753