Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning
An inference-time meta-reasoning controller lifts long-horizon agent scores, reaching 71.5% on ProgramBench with GPT-5.5.
The paper introduces agentic meta-reasoning, an inference-time harness where workers do task computation and a controller decides what to continue, restart, or stop. The controller keeps a compact account of the run instead of replaying full history and dispatches work from persistent memory. On ProgramBench it scores 71.5% with GPT-5.5 versus 58.0% for Codex, and 67.2% with Opus 4.8 versus 65.5% for Claude Code. Across other long-horizon benchmarks it gains 3.6 to 4.2 points over direct control, though overhead can hurt at small budgets.
- A controller picks next work from a compact run account under a remaining budget.
- GPT-5.5 meta-reasoning scores 71.5% on ProgramBench versus 58.0% for Codex.
- Opus 4.8 reaches 67.2% versus 65.5% for Claude Code.
- Other benchmarks gain 3.6 to 4.2 points over direct control.
- Overhead can reduce gains when the compute budget is small.
Full article258 words · extracted from arxiv.org · click to collapse
As agents take on longer and more complex problems, controlling the execution becomes a task in its own right. Each step in the run brings new control choices, like which partial work to build on, whether to start fresh, or when to stop. We introduce agentic meta-reasoning, an inference-time harness that makes these choices an explicit and structured reasoning process. Workers carry out the task-level computation, while a controller consolidates what the run has established, explores next options, assesses what each option is worth under the remaining budget, and dispatches the chosen work with context drawn from persistent memory. Between decisions the controller carries only a compact account of the run rather than replaying its full history. Our baselines span production coding agents and research harnesses, together with a Direct Control Agent using the same workers and compute budget allowance. On ProgramBench, which tests long-horizon agentic capability through program reconstruction, meta-reasoning achieves 71.5% with GPT-5.5 against 58.0% for Codex; with Opus 4.8 it achieves 67.2% against 65.5% for Claude Code. On the other benchmarks, spanning abstract reasoning, multi-domain long-horizon reasoning, and proof generation, it gains between 3.6 and 4.2 points over direct control, averaged across three frontier models. It keeps improving over the tested budget ranges where direct control plateaus, though its overhead can hurt at small budgets. Artifact-graph analysis reveals more reuse of earlier work, higher coverage of correct solutions in most settings, and nonuniform gains in final selection. These results indicate that spending computation on structured control becomes more important as agents scale to longer runs.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.38147