VepAgent: Bridging Causal-Transition via Tool-Augmented Reinforcement Learning for Video Event Prediction
VepAgent uses tool-augmented reinforcement learning to predict video events by modeling unobserved causal transitions.
VepAgent is an agentic framework for video event prediction that combines causal-transition reasoning with tool-augmented reinforcement learning. The authors build futurebench-4K, a chain-of-thought set for supervised fine-tuning of unobserved intermediate states, and a tool library for state tracking, frame retrieval, and region magnification. A composite reward optimizes prediction accuracy, causal coherence, and reliance on visual evidence rather than text similarity. Evaluations on FutureBench and NEPBench report state-of-the-art results over larger multimodal models.
- VepAgent models unobserved causal transitions for video event prediction.
- futurebench-4K supplies chain-of-thought data for supervised fine-tuning.
- Tools cover state tracking, frame retrieval, and region magnification.
- A composite reward targets accuracy, causal coherence, and visual grounding.
- The method reports state-of-the-art results on FutureBench and NEPBench.
From the original article
1 min read at huggingface.co
Multimodal Large Language Models (MLLMs) have demonstrated remarkable potential in video understanding, yet their reliance on retrospective summarization and text-centric priors often limits their ability to bridge unobserved causal transitions when applied to Video Event Prediction (VEP).
By Qiutong Chen, Yuchan Guo, Zhenlong Yuan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.