TCoVP grounds video reasoning segmentation in timestamped visual primitives
Original title: Think Visually Across Space and Time: Grounded Reasoning for Video Segmentation
The method builds a chronological chain of entity references and spatial evidence, then uses selected primitives as box prompts for mask prediction.
Video reasoning segmentation must identify an implicitly described target and track its pixel mask while resolving spatial and temporal relations. Text-only chain-of-thought can justify a choice without showing the frames that support it. TCoVP interleaves language with timestamped entity references and spatial evidence, then feeds selected primitives into video mask prediction. Training uses TCoVP-SFT-15k and TCoVP-RL-13k; the authors report gains on five benchmarks across three backbones and say models and code will be released.
- TCoVP-SFT-15k and TCoVP-RL-13k train the grounded reasoning chain.
- Experiments span five video reasoning segmentation benchmarks and three backbones.
- Selected primitives supply box prompts and temporal support for mask prediction.
- The authors say models and code will be publicly released.
From the original article
1 min read at huggingface.co
Video Reasoning Segmentation (VRS) requires a model to identify an implicitly described target and predict its pixel-level mask trajectory, while resolving the spatial relations and temporal events that determine the answer.
By Junwei Wen, Deshui Miao, Guangming Lu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.