When Left Is Right? Camera-Motion Understanding beyond Screen
CameraVLM, a 9B model, judges camera motion better than prior VLMs and matches humans.
CameraVLM is a 9B vision-language model for understanding off-screen camera motion, which most VLMs judge no better than chance. It combines camera-motion tokens with on-screen flow tokens and uses two-stage training so the language model interprets both together. Across three benchmarks and more than 20 baselines, it outperforms open-source and camera-motion-specific models by a wide margin. It matches or surpasses GPT-6 Astra and human viewers in a 48-participant study.
- Most VLMs perform near chance when judging whether a camera moved left or right.
- CameraVLM pairs off-screen camera tokens with on-screen flow tokens.
- Two-stage training keeps language alignment in the loop.
- The 9B model beats more than 20 baselines and matches or exceeds GPT-6 Astra and humans.
- Products
- CameraVLM
- AI models
- CameraVLMGPT-6 Astra
From the original article
1 min read at huggingface.co
Can a vision-language model tell left from right? Modern VLMs solve visual puzzles and follow hour-long videos, yet when asked whether the camera moved left or right, most do no better than a coin flip. Adding camera-motion data or geometric representations has been tried, separately, and neither closes the gap.
By Chong Cheng, Ran Zhang, Xinyuan Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.