SleuthVLM outperforms Gemini on omni video benchmarks
Original title: SleuthVLM: Do We Really Need Audio-in-Video for Omni Video Understanding?
SleuthVLM reaches 75.0% accuracy on omni benchmarks via transcription and agentic reasoning.
SleuthVLM is an agent harness on a VLM that transcribes speech to text and sleuths video without direct audio. It achieves 75.0% average accuracy on four omni benchmarks with videos up to 90 minutes, beating Gemini 3.8 Flash by 2.6 points and topping Video-MME-v2 at 77.8%. With an open-source 27B VLM it outperforms all other open-source omni models.
- SleuthVLM scores 75.0% on omni benchmarks
- Tops Video-MME-v2 leaderboard with 77.8%
- Agentic ability beats larger models as bottleneck
- No direct audio needed for strong performance
- Organizations
- Hugging Face
- AI models
- SleuthVLMGemini 3.8 Flash
From the original article
1 min read at huggingface.co
Omni models answer questions about videos by jointly processing sampled frames and audio in one turn, with progress driven largely by scaling model size and omni training data.
By Peiran Wu, Fangfei Gou, Zhaoxi Mu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.