CoEvoWhen: Policy-Tool Coevolution for Ultra-Long Video Temporal Grounding
CoEvoWhen jointly evolves VLM policies and media tools for ultra-long video temporal grounding without updating parameters.
Ultra-long video temporal grounding must balance long-range evidence search with fine-grained event understanding under a limited visual budget. CoEvoWhen jointly evolves high-level policies and executable media tools from a vision-language model's reasoning trajectories, producing a reusable skill without updating model parameters. An external skill updater refines orchestration of image- and video-based observations and can upgrade or create tools for long-video evidence acquisition. Across five benchmarks and three VLMs, the evolved skill improved grounding accuracy, reduced visual token cost, and transferred to general long-video question answering without further task-specific evolution.
- CoEvoWhen evolves policies and tools from VLM trajectories without parameter updates.
- An external updater refines long-range image and fine-grained video observations.
- The updater can upgrade existing media tools or create new ones.
- Five benchmarks and three VLMs showed better accuracy and lower token cost.
- The evolved skill transferred to general long-video QA without extra evolution.
From the original article
1 min read at huggingface.co
Ultra-long video temporal grounding requires balancing long-range evidence search with fine-grained event understanding under a limited visual budget, yet existing agentic methods still rely largely on predefined policies and tool capabilities.
By Yiduo Jia, Muzhi Zhu, Jinchuan Shi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.