SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance
SAGE uses structural guidance to cut long-horizon reasoning biases, with up to an 8-fold gain.
SAGE (Structural Admissibility-Guided Exploration) addresses exploration bias and compounding bias in long-horizon LLM reasoning under sparse rewards. It combines algebraic sparsification, which limits spurious branching, with hyperbolic structural guidance that supplies dense depth-wise signals. Across 12 benchmarks and seven model families, SAGE outperformed competitive baselines, including up to an 8-fold improvement on the Andrews-Curtis problem. The authors released code on GitHub.
- SAGE targets exploration bias and compounding bias under sparse rewards.
- Guidance combines algebraic sparsification with hyperbolic depth-wise signals.
- Evaluated on 12 benchmarks across seven model families.
- Reports up to an 8-fold gain on the Andrews-Curtis problem.
Full article204 words · extracted from arxiv.org · click to collapse
Long-horizon reasoning remains a central challenge for large language models (LLMs) under sparse-reward regimes. We argue that this brittleness arises from two biases induced by complex reasoning spaces: an exploration bias, where models are drawn toward locally plausible but structurally unstable branches, and a compounding bias, where small local deviations accumulate across depth and suppress rare rewards. We introduce Symbolic Closure Analysis (SCA) as a theoretical lens characterizing how branching structures and sparse rewards induce these biases in long-horizon reasoning with local admissibility, and as a design principle for structural priors in less formal reasoning tasks. Motivated by this analysis, we propose SAGE (Structural Admissibility-Guided Exploration), a unified framework that injects structural guidance to alleviate exploration bias and compounding bias in long-horizon reasoning. SAGE combines two complementary structural guidance: algebraic sparsification, which projects locally admissible candidates onto operator-indexed algebraic subspaces to suppress spurious branching and mitigate exploration bias, and hyperbolic structural guidance, which embeds reasoning states into a negatively curved space to provide dense depth-wise signals and mitigate compounding bias. Across 12 benchmarks and 7 model families, SAGE outperforms competitive baselines. In particular, SAGE achieves up to an 8-fold improvement on the Andrews-Curtis problem, an open real-world long-horizon task. Code is available at: https://github.com/Susan571/SAGE-NeurIPS2026.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.30192