Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models
Researchers extract hidden chain-of-thought from frontier models, including GPT-6 Astra, matching native reasoning quality.
The paper shows a custom API tool can induce closed frontier models to externalize intermediate reasoning. Extracted traces match native chain-of-thought on open models and beat no-reasoning baselines on competition math, science, and code. GPT-6 Astra is described as token-efficient, selecting a correct trajectory early and externalizing only crucial steps.
- A custom API tool externalizes hidden reasoning in closed models.
- Extracted traces match native CoT and beat no-reasoning baselines.
- Tests cover competition mathematics, science, and code generation.
- Astra commits early and externalizes only crucial reasoning steps.
Full article165 words · extracted from huggingface.co · click to collapse
The rapid capability gains of frontier language models are widely attributed to improved reasoning abilities, yet this cannot be verified as raw CoT traces in closed-source systems are hidden. By registering a simple custom tool through a standard API feature, we induce frontier models to externalize intermediate reasoning. Because these traces may reflect post-hoc rationalization rather than genuine reasoning, we first evaluate against native CoT on open-source models and extend to closed-source frontier models including GPT-6 Astra. We find that the extracted reasoning matches native reasoning performance and substantially outperforms no-reasoning baselines, across competition mathematics, science, and code generation. We then characterize how frontier models structure their intermediate reasoning. Across token efficiency, reasoning-step types, and induced reasoning trees, we identify systematic differences in how models externalize, compress, and organize reasoning. We find that Astra exhibits token-efficient directed reasoning, selecting a correct trajectory earlier, while resolving elementary steps internally and externalizing only crucial reasoning. These findings provide a behavioral lens on frontier-model reasoning beyond benchmark scores.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.26637