VideoMemBench measures MLLM in-context video memory
Original title: VideoMemBench: How Well Do MLLMs Remember in Video Conversations?
VideoMemBench pairs videos with multi-turn dialogues containing episodic and answer dependencies to test memory recall in MLLMs.
Multimodal large language models process hour-long videos for extended multi-turn dialogue requiring memory of prior content. Existing benchmarks lack cross-turn dependencies so VideoMemBench introduces them with necessity and non-leakage verification. It includes 326 videos and 2,476 turns with 1,387 dependencies. Claude Opus 5 scores highest at 45.7% revealing challenges in in-context video memory.
- Episodic and Answer Dependencies verified
- 3 LLM roles: planner generator judger
- Human expert review of dialogues
- 326 videos 2,476 turns
- Claude Opus 5 scores 45.7% highest
- AI models
- Claude Opus 5
From the original article
1 min read at huggingface.co
Multimodal large language models (MLLMs) can now process hour-long videos, making extended multi-turn dialogue over a single video a natural mode of use. Sustaining such a dialogue requires more than answering each question in isolation: the model must retrieve and reuse information established earlier in the conversation, both what it has seen and what it has said, an ability we term in-context video memory.
By Zhengdong Hu, Chao Wang, Kaituo Feng · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.