PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving
PlannerForge unifies scenario-based testing of autonomous driving motion planners in one LLM-agent framework, outperforming prior baselines.
PlannerForge is an LLM-agent framework that covers the full scenario-based testing pipeline for autonomous driving systems, spanning scenario generation, selection, modification, routing, planner testing, plus new enhancement and benchmarking stages. In evaluations with 10 off-the-shelf LLMs, best-per-task scores range from 0.88 to 1.00, and open-source 20-35B backends such as Qwen3.6:35B match commercial APIs on most tasks. End-to-end chaining retains 83% (commercial) and 78% (open) of seed queries, beats Scenario Factory 2.0 on executable generation, and cost-tuning lifts planner success from 50.4% to 70.2% while cutting collisions from 19.0% to 8.4%.
- Unified LLM-agent framework for end-to-end autonomous driving scenario testing
- Open-source 20-35B models match commercial APIs on most tasks
- Beats BM25 rank-1 selection (92.0% vs 67.5%) and prior edit validity (>=94% vs 31%)
- Cost-tuning cuts planner collisions from 19.0% to 8.4% at N=400
- Outperforms Scenario Factory 2.0 with 193 vs 144 executable scenarios
- Products
- PlannerForge
- AI models
- Qwen3.6:35B
From the original article
1 min read at huggingface.co
Ensuring the safety of autonomous driving is a critical challenge. Scenario-based testing is a systematic process used to validate Autonomous Driving Systems (ADSs), but it remains a fragmented modular pipeline in which scenario generation, retrieval, modification, ADS execution, and results analysis are performed by separate tools with little interaction.
By Yuan Gao, Sebastian Müller, Mattia Piccinini · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.