OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction
OneStreamer-4B unifies streaming video perception, caption memory, and proactive responses, topping eight benchmarks.
OneStreamer jointly learns query-independent evidence recording and task response for streaming video. Proactive Hierarchical Caption Memory stores time-grounded captions and event summaries so the model need not revisit historical visual features. Proactive State Transition Learning supervises only 27.5% of annotated state tokens yet beats dense state supervision. Trained with the OneStreamer-1M dataset of over one million records, the 4B model led all compared methods on eight streaming video understanding benchmarks.
- The 4B model led every compared method on eight benchmarks.
- Caption memory avoids revisiting historical visual features at inference.
- PSTL uses 27.5% of state tokens yet beats dense supervision.
- OneStreamer-1M holds more than one million streaming records.
From the original article
1 min read at huggingface.co
Streaming video LLMs must retain evidence before its relevance to future tasks is known and respond when sufficient evidence becomes available. The challenge is to form reusable factual memory without compromising real-time perception. We introduce OneStreamer, which jointly learns query-independent evidence recording and task response through a shared proactive generation process.
By Xiangyu Zeng, Yuandong Yang, Zhiqiu Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.