Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models
Researchers extract hidden chain-of-thought from frontier models, including GPT-6 Astra, and compare reasoning styles.
Researchers report that registering a simple custom tool through a standard API can induce frontier models to externalize intermediate reasoning that is otherwise hidden. On open-source models, extracted traces match native chain-of-thought performance and substantially beat no-reasoning baselines on competition mathematics, science, and code generation. Extending the method to closed models, including GPT-6 Astra, they find Astra selects a correct trajectory earlier, resolves elementary steps internally, and externalizes fewer tokens.
- A custom API tool induces frontier models to externalize hidden chain-of-thought.
- Extracted traces match native CoT and beat no-reasoning baselines.
- Tasks cover competition math, science, and code generation.
- GPT-6 Astra is described as more token-efficient, externalizing only key steps.
Full article165 words · extracted from arxiv.org · click to collapse
The rapid capability gains of frontier language models are widely attributed to improved reasoning abilities, yet this cannot be verified as raw CoT traces in closed-source systems are hidden. By registering a simple custom tool through a standard API feature, we induce frontier models to externalize intermediate reasoning. Because these traces may reflect post-hoc rationalization rather than genuine reasoning, we first evaluate against native CoT on open-source models and extend to closed-source frontier models including GPT-6 Astra. We find that the extracted reasoning matches native reasoning performance and substantially outperforms no-reasoning baselines, across competition mathematics, science, and code generation. We then characterize how frontier models structure their intermediate reasoning. Across token efficiency, reasoning-step types, and induced reasoning trees, we identify systematic differences in how models externalize, compress, and organize reasoning. We find that Astra exhibits token-efficient directed reasoning, selecting a correct trajectory earlier, while resolving elementary steps internally and externalizing only crucial reasoning. These findings provide a behavioral lens on frontier-model reasoning beyond benchmark scores.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.26637