Qwen-Drive 1.0 tells you why it brakes, just don't expect the explanation to match the maneuver
Alibaba's Qwen-Drive 1.0 adds 3D perception and planning modules to Qwen3.5-4B for driving tasks, though explanations often mismatch maneuvers.
Qwen-Drive 1.0, built on Qwen3.5-4B, combines spatial perception, traffic question answering, and route planning in one vision-language model, adding a bird's-eye-view perception module and a Planning Expert trained via staged fine-tuning and reinforcement learning. The paper finds text-image models do not inherently grasp 3D space; spatial accuracy only improved when the base vision-language model itself was trained on spatial tasks, while avoiding catastrophic forgetting of general knowledge. The cut reinforcement learning-trained version halved road-departure rate in simulation from 24% to 12%, and the model beats specialized driving models in most of Qwen's benchmarks, but its explanations sometimes conflate causes like distant red lights and crossing children, and results partly rest on self-designed tests. The work follows prior findings from PaLM-E and a UC Santa Cruz adversarial sign attack on DriveLM showing VLM driving models' reasoning and spatial gaps.
- Built on Qwen3.5-4B with added 3D perception and Planning Expert modules
- RL training cut simulated road-departure rate from 24% to 12%
- Spatial understanding must be trained deliberately; general knowledge retained
- Explanations can conflate causes; metrics partly rely on self-designed benchmarks
Full article1,011 words · extracted from the-decoder.com · click to collapse
Qwen-Drive 1.0 handles three tasks in one AI model: spatial perception of the environment, answering questions about traffic, and route planning. The researchers confirm that a text-image model doesn't automatically understand three-dimensional space just because it can describe pictures.
Existing driving models take a general text-image model and fine-tune it on driving data, mostly question-and-answer pairs about traffic situations. According to the paper, this approach has two weaknesses. A model trained mainly on traffic Q&A still can't reliably detect distances, positions, and open spaces. And if it becomes too specialized on driving data, it loses the broad general knowledge from its original training through what the researchers call "catastrophic forgetting," which is exactly the kind of knowledge that matters most in rare, unexpected traffic situations. The model Alibaba's research division built is supposed to address both problems.
A separate module checks whether the model actually understands space
Qwen-Drive-1.0 builds on Qwen3.5-4B, released in February, and adds two extra components. The first generates a bird's-eye-view map of the surroundings by spotting objects in 3D space, figuring out which areas are occupied, and tracing the road layout. The researchers say it doubles as a measuring tool that reveals how much spatial information the model is actually pulling from the images.
The second component, the Planning Expert, uses internal model data to plan the car's future movement. When the researchers trained only the added component and left the vision-language model untouched, spatial accuracy stayed low, confirming that a model capable of describing images in detail doesn't automatically grasp three-dimensional space. Only when the team also trained the vision-language model itself on spatial tasks did performance improve significantly, meaning the ability to spatially understand traffic scenes has to be built in deliberately.
Training starts with the perception module, then combines perception and question answering, followed by route planning. The final step refines the model's behavior through reinforcement learning. For the vision-language component, the team combined 24 publicly available datasets of traffic scenes. These datasets had different structures and sometimes contained errors. An AI model standardized the questions and answers and aligned them with the original data. The team also built its own examples explaining why the car should make a specific driving decision, like which object triggers braking.
One model for both the cockpit and the driving system
In modern vehicles, the infotainment system and the driving system are converging on a single computing unit instead of running on two separate controllers. The authors argue that a model trading general capabilities for pure driving performance doesn't help here, because the cockpit would still need its own model and extra compute for tasks like dialog or open-ended questions.
Qwen-Drive 1.0 scores well above the unmodified base model Qwen3.5-4B on questions about traffic scenes, according to the paper. The biggest gap shows up when the model has to explain cause and effect, like why the car should brake or turn. General knowledge holds up too: The model shows almost no drop on tests outside of driving and even scores slightly higher on some spatial tasks.
From simulation to the road
For driving planning, the team tested the model at several difficulty levels, from simple predictions up to a simulator where errors compound over time. In the simulator, the version retrained with rewards cut the rate at which the car veered off the road from 24 to 12 percent. It also drove more cautiously and covered less distance overall.
The model's explanations don't always pinpoint the actual cause of a situation, though. A red light in the distance and a child stepping into the road call for very different reaction times, and the model can conflate the two. The planned maneuver also doesn't always match the reasoning the model gave beforehand. Some results rest on test procedures the authors designed or rebuilt themselves, so individual metrics say little about how the system would handle messy real-world driving.
The Qwen team measured this gap in spatial understanding itself using its HopChain benchmark. Here, vision-language models misclassified objects and confused spatial relationships even while scoring well on image-text benchmarks. Catastrophic forgetting is a familiar problem, too. When Google Deepmind built PaLM-E in 2023, a model for language, images, and robot control, the smaller variants lost a big chunk of their language ability after robot training. The largest version, at 562 billion parameters, lost almost nothing.
Pairing a language model with a driving function opens up a new attack surface. Researchers at UC Santa Cruz placed a labeled sign in the camera's field of view and tricked the DriveLM driving system into swerving toward crossing pedestrians, even though it had detected them correctly. Research is meanwhile shifting toward World Action Models, which also predict how the environment changes in response to the agent's own actions.
The Qwen team is releasing the model to the research community for free on Hugging Face, ModelScope, and GitHub.
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.
Text extracted automatically; images, tables and formatting may be missing. Original: https://the-decoder.com/qwen-drive-1-0-tells-you-why-it-brakes-just-dont-expect-the-explanation-to-match-the-maneuver/