SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance
SAGE uses structural guidance to cut long-horizon reasoning biases, with up to an 8-fold gain.
SAGE (Structural Admissibility-Guided Exploration) addresses exploration bias and compounding bias in long-horizon LLM reasoning under sparse rewards. It combines algebraic sparsification, which limits spurious branching, with hyperbolic structural guidance that supplies dense depth-wise signals. Across 12 benchmarks and seven model families, SAGE outperformed competitive baselines, including up to an 8-fold improvement on the Andrews-Curtis problem. The authors released code on GitHub.
- SAGE targets exploration bias and compounding bias under sparse rewards.
- Guidance combines algebraic sparsification with hyperbolic depth-wise signals.
- Evaluated on 12 benchmarks across seven model families.
- Reports up to an 8-fold gain on the Andrews-Curtis problem.
From the original article
1 min read at arxiv.orgLong-horizon reasoning remains a central challenge for large language models (LLMs) under sparse-reward regimes. We argue that this brittleness arises from two biases induced by complex reasoning spaces: an exploration bias, where models are drawn toward locally plausible but structurally unstable branches, and a compounding bias, where small local deviations accumulate across depth and suppress rare rewards.
By Xinyue Zeng · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.