AI models' written reasoning steps correspond to distinct internal patterns, a new study finds
KAIST and Naver AI Lab researchers show LLM reasoning steps like extraction and computation map to distinct activation patterns, strongest in middle layers.
Researchers at KAIST and Naver AI Lab defined eight recurring reasoning operations, including extraction, decomposition, formula recall, deduction, and computation, and showed they correspond to separable activation patterns in Qwen2.5-7B, Qwen3-8B, and Gemma4-31B on math tasks, with GPT-5 labeling solution segments. The separation peaks in middle layers, holds even when a computation step produces a wrong answer, and goes beyond surface-level token choice. Findings replicated on Llama-3-8B, and classifiers trained on Qwen3-8B transferred to GPQA-Diamond and MATH-500. The authors note that using internal states for error detection or mid-generation steering remains future work.
- Eight reasoning operations are reliably distinguishable in model activations across three tested models.
- Separation is strongest in middle layers and exceeds what token choice alone predicts.
- Blocking attention to the preceding 30 tokens weakens operation signals, showing contextual dependence.
- Findings replicated on Llama-3-8B and transferred to GPQA-Diamond and MATH-500 benchmarks.
- Chain-of-thought oversight is limited; Anthropic found models disclose used hints in only 25-39% of cases.
- Tags
- activation-analysischain-of-thoughtinterpretabilitykaistllama-3-8bllm-reasoningnaver-ai-labqwen3-8b
- Organizations
- KAISTNaver AI Lab
From the original article
3 min read at the-decoder.com
Reasoning steps like calculation, formula retrieval, and deduction are clearly separable in a model's internal states, especially in the middle layers. That matters for AI safety, because models process more than their visible chain of thought reveals. The article AI models' written reasoning steps correspond to distinct internal patterns, a new study finds appeared first on The Decoder.
By Jonathan Kemper · the-decoder.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.