SOMA: A Benchmark for Fine-Grained Egocentric Behavior and Body-Motion Understanding
SOMA benchmark tests 37 MLLMs on egocentric body-motion understanding with 1,632 questions across configuration, dynamics, and temporal reasoning
SOMA comprises 1,632 human-verified questions from egocentric video covering body configuration, motion dynamics, and temporal reasoning. Humans reach 68.5% accuracy while the top MLLM Qwen3.6-27B scores 50.0% zero-shot. Analysis reveals common errors like misreading the wearer's body or timing biases. Introduces aligned training dataset and multimodal baseline.
- 1,632 questions across three capability axes
- 37 MLLMs evaluated including closed- and open-source
- Qwen3.6-27B at 50.0% accuracy
- Human baseline at 68.5% from video alone
- AI models
- Qwen3.6-27B
From the original article
1 min read at huggingface.co
Perceiving human body motion is critical for multimodal systems that understand and assist people. Yet current evaluations of Multimodal Large Language Models (MLLMs) largely focus on objects, scenes, and coarse actions, leaving their ability to reason about the body itself poorly assessed.
By Matea Tashkovska, Naël Ouerghemi, Haozhe Qi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.