Event-STVG benchmark tests long-form event reasoning in video
Original title: Event-STVG: Spatio-Temporal Video Grounding for Long-Form Event Reasoning
The strongest tested MLLM reached only 5.4% m-vIoU at a 10-minute horizon.
Researchers introduce Event-STVG, moving spatio-temporal video grounding from instance descriptions toward event-level reasoning. The task combines long-form context, repeated-event reasoning, localization, role inference, and multi-target grounding. The benchmark contains 4,016 annotated event occurrences across soccer and seven other domains. Tested MLLMs often confuse repeated events and fail spatial or joint role grounding.
Unconfirmed: The 5.4% m-vIoU result comes from the authors' evaluation of existing MLLMs.
- Annotations span soccer and seven additional domains.
- Models rarely ground every required role at once.
- Code and annotations are at github.com/THU-KEG/EventSTVG.
- Products
- Event-STVG
- Organizations
- THU-KEG
From the original article
1 min read at huggingface.co
Spatio-Temporal Video Grounding (STVG) aims to jointly localize a target in space and time given a natural-language query. Despite continued advances, existing STVG benchmarks remain largely centered on localizing targets through instance-specific descriptions, leaving them largely decoupled from long-form reasoning increasingly emphasized in multimodal large language models (MLLMs).
By Yizhe Zhang, Xujia Wang, Songqi Tang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.