Frontier VLMs judge 3-D relations from picture-plane cues
Original title: CueClash: Frontier Vision-Language Models Judge 3-D Relations Close to the Picture Plane
The shortcut gap is +18 to +37 points for Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash at low reasoning effort.
CueClash renders scenes so the 2-D cue for each of five spatial relations is correct on exactly half the statements. Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash at low reasoning effort score 18 to 37 points higher when that cue is right. Given ground coordinates and relation definitions they answer 99 to 100 percent correctly, and the failure is traced to going from the picture to the layout. The same models also lose accuracy on CV-Bench photographs where a reliable 2-D cue misleads.
- Dropping two confounded relations leaves a shortcut gap of +7 to +29 points.
- Pixel measurements with camera depths remove most of the gap for objects fitting openings.
- Opus 5.5 and Gemini maps undo about 0.6 of the picture's depth shortening.
- GPT-5.5's overhead maps stay close to a copy of the picture.
From the original article
1 min read at arxiv.orgA vision-language model (VLM) can judge 3-D relations from the picture or from the ground layout. In photographs the two usually agree. We separate them with CueClash, rendered scenes in which the 2-D cue for each of five relations is right on exactly half the statements.
By Soham Dan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.