Visible-State Supervision targets under-specified streaming video recognition
Original title: Supervising the Visible State: Separating Representation from View Selection in Streaming Video
Under a matched recent-frame setup, ViSS improves OVO-Bench and helps hallucination detection most, while staying comparable on StreamingBench.
The paper argues that shared action labels leave event order, accumulation and relations unconstrained, a problem it calls state under-specification. Visible-State Supervision trains on source-grounded probes of identity, order, accumulation, relation change and answer sufficiency. Test-Time Causal View Search then widens evidence from recent frames to sparse event and compact history views without changing the learned model. Together, the authors say the methods reach the best accuracy among systems they compare on StreamingBench and OVO-Bench while processing fewer frames than exhaustive selection.
Unconfirmed: The best-accuracy claim is limited to the published systems compared in the paper.
- Supervision helps hallucination detection and causal reasoning in their split of gains.
- View search helps event recall and forward reasoning.
- Replay shows most of the remaining oracle gap is a correct answer later replaced.
- TVS keeps the learned model fixed and changes only evidence access.
From the original article
1 min read at huggingface.co
An action seen once and the same action seen three times receive the same action-recognition label, so that supervision does not require a model to count. Answers shared across histories that differ in event order, accumulation, or relations leave those distinctions unconstrained; we call this state under-specification.
By Yihao Hu, Pan Wang, Yitong Lin · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.