Compacting reasoning history shortens future generation and can cut agent cost
Original title: Why Pay for Long Reasoning When Rewrite Do Trick? Causal Effects of Reasoning History Compaction on Future Generation and Agent Cost
Open-weight models rewrite earlier thoughts into compact state records before tool results, without retraining.
The paper tests whether open-weight models can reason more briefly by rewriting their own earlier thoughts into compact state records before pending tool results are revealed. Continuations from accepted compact histories usually use fewer reasoning tokens than original histories. A coding case study found less repeated verification after a shared candidate solution. In software-agent deployments, compaction can lower total spending, though quality and rewrite-inclusive time depend on the target and compressor.
Unconfirmed: Specific models, spending reductions, and quality effects are not quantified in the abstract.
- The rewrite occurs before pending tool results are revealed.
- No retraining is required; only the form of earlier thoughts changes.
- Accepted compact histories usually shorten future reasoning across tested models.
- A coding case study showed less repeated verification after a shared solution.
- Authors recommend model-specific policies checked for efficiency and retained capability.
From the original article
1 min read at huggingface.co
Reasoning substantially improves language models' ability to solve difficult problems, but long reasoning traces make these gains expensive. Efficient reasoning should express useful analysis in fewer tokens by reducing repetition and text that adds little value to solving the problem. Progress in proprietary models motivates this goal, yet their hidden scratchpads limit direct study of how it is achieved.
By Anton Hristov · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.