SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance
SAGE uses algebraic and hyperbolic guidance to reduce long-horizon LLM reasoning biases, with up to an 8-fold gain.
SAGE (Structural Admissibility-Guided Exploration) targets exploration and compounding biases in long-horizon LLM reasoning under sparse rewards. It combines algebraic sparsification, which projects locally admissible candidates onto operator-indexed subspaces, with hyperbolic embeddings that provide dense depth-wise signals. Across 12 benchmarks and seven model families it outperforms competitive baselines, including up to an 8-fold improvement on the Andrews-Curtis problem. The paper also introduces Symbolic Closure Analysis as a theoretical account of how branching structure and sparse rewards create those biases.
- Targets exploration bias and compounding bias in sparse-reward reasoning.
- Combines algebraic sparsification with hyperbolic structural guidance.
- Evaluated across 12 benchmarks and seven model families.
- Reports up to an 8-fold gain on the Andrews-Curtis problem.
From the original article
1 min read at huggingface.co
Long-horizon reasoning remains a central challenge for large language models (LLMs) under sparse-reward regimes. We argue that this brittleness arises from two biases induced by complex reasoning spaces: an exploration bias, where models are drawn toward locally plausible but structurally unstable branches, and a compounding bias, where small local deviations accumulate across depth and suppress rare rewards.
By Xinyue Zeng, Jiawei Zhang, Yujun Yan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.