Adversarial Speculative Reasoning Improves Long Horizon Agents
Training draft models to reject at critical decision points improves agent accuracy by training 0.06% to 0.28% of parameters on agentic coding benchmarks.
Speculative decoding affects long-horizon agent reasoning through numerical noise that grows within verification windows. The paper trains draft models to be rejected at key points identified in agent traces, ensuring critical tokens are sampled from the target model under least noisy conditions. This recasts speculative decoding as an allocator for context precision and raises end-to-end success rate on complex agentic coding benchmarks.
- Numerical noise grows within verification windows after 30 steps.
- Draft acceptance can change output distribution after 30 steps.
- Critical reasoning tokens encoded with highest precision.
- Only 0.06% to 0.28% of parameters trained for improvement.
From the original article
1 min read at arxiv.orgSpeculative decoding has been treated purely as a systems optimization that speeds up large language model decoding without changing the model behavior. Under this view, the growing effort to train better draft models has been focused solely on acceptance rate. We argue that this view does not hold true for long-horizon agent reasoning.
By Hongyin Luo · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.