Do LLM Agents Execute the Plans They Declare? From Planning-Mode Declaration to Pattern-Specific Execution
Pattern-specific executors close LLM agents' plan-execution gap, lifting ALFWorld success from 0.48 to 0.92.
The paper studies the Plan Declaration–Execution Gap: LLMs declare a planning mode, but generic Plan+ReAct often fails to follow it. A deterministic Planning-as-Routing setup dispatches tasks to Predefined, Sequential, Hierarchical, or Search executors. Across four benchmarks and three LLMs, pattern-specific execution lifts ALFWorld success from 0.48 to 0.92 and SWE-bench Verified from 0.36 to 0.44, while reliable mode selection remains unsolved.
- Planning-as-Routing declares Predefined, Sequential, Hierarchical, or Search modes.
- Only 22–45% of Plan+ReAct trajectories preserve the declared structure.
- Pattern-specific executors raise ALFWorld success from 0.48 to 0.92.
- Search is strongest on ALFWorld; Hierarchical on SWE-bench Verified.
- Models rarely pick the best mode; few-shot selection helps sometimes.
From the original article
1 min read at arxiv.orgLarge language models (LLMs) enable agents to solve long-horizon tasks by generating a plan and then executing it in an environment. However, successful planning requires two distinct capabilities: selecting an appropriate plan for the task and executing it faithfully. Existing planner--executor systems can fail at either stage, while final task success alone cannot distinguish selection from execution failures.
By Subba Reddy Oota · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.