AnchorReasoning: A Visual Grounding and Causal Reasoning Dataset in Long-Tail Autonomous Driving Scenarios
AnchorReasoning provides visually grounded reasoning data for long-tail autonomous driving across 416,119 annotated frames.
AnchorReasoning is a visually grounded reasoning dataset for long-tail autonomous driving, built on WOD-E2E. It contains 416,119 annotated frames and 395,379 decision-critical elements across four major categories and 19 fine-grained types. Each frame is a visually grounded chain-of-thought covering element identification, attributes, action rationale, and trajectory planning. Curriculum fine-tuning across eight backbones cut 5-second ADE and FDE by 7.84 and 11.86 while using 18.5 fewer reasoning tokens and 0.32 seconds less inference per frame.
- Built on WOD-E2E with 416,119 frames and 395,379 decision-critical elements.
- Frames use visually grounded chain-of-thought linking evidence, rationale, and planning.
- Across eight backbones, 5-second ADE fell 7.84 and FDE fell 11.86.
- Models used 18.5 fewer reasoning tokens and 0.32 seconds less latency per frame.
Full article179 words · extracted from arxiv.org · click to collapse
Vision-language models (VLMs) offer a promising approach to long-tail autonomous driving, but existing driving datasets provide limited supervision for connecting decision-critical visual evidence with reasoning and planning. We introduce AnchorReasoning, a visually grounded reasoning dataset built on WOD-E2E, containing 416,119 annotated frames and 395,379 decision-critical elements across four major categories and 19 fine-grained types. Each frame is organized as a visually grounded chain-of-thought (VG-CoT) that links decision-critical element identification and localization, element attributes and implications, driving-action rationale, and action and trajectory planning. We further develop a curriculum supervised fine-tuning strategy that progressively learns these hierarchical capabilities, together with an object-size-aware grounding metric for evaluating localization quality. Experiments across eight general-purpose, embodied-AI, and AV-specific backbones show that VG-CoT supervision improves grounded reasoning and trajectory prediction. Across models, 5-s ADE and FDE decrease by 7.84 and 11.86, while RFS Frame and Cluster improve by 1.66 and 1.70. These gains are achieved with 18.5 fewer reasoning tokens and 0.32 s/frame lower inference latency on average, demonstrating the value of visually grounded, decision-focused supervision for VLM reasoning and planning in long-tail autonomous driving.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.28366