TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understanding
TRACE benchmarks streaming video models on timing, triggers, workload, and reliability, not accuracy alone.
TRACE is a condition-aware benchmark for streaming video understanding that records when evidence becomes valid, how visual history is processed, and when responses fire. It pairs temporally audited tasks with a causal Core-Adapter protocol and reports answer quality, timeliness, false alarms, missed windows, workload, and reliability. On 1,240 records from 517 videos, eight public models in eight configurations show that near-identical QA accuracy can hide large gaps in completion, validity, and generation cost.
- Benchmark covers 1,240 records drawn from 517 videos
- Eight public models evaluated across eight configurations
- Similar QA scores mask differences in completion and workload
- Proactive behavior splits into delay, false alarms, and missed windows
- Code is released at github.com/om-ai-lab/trace-bench
From the original article
1 min read at huggingface.co
Streaming video understanding requires models to interpret evidence as it arrives, yet current evaluations often report task scores without specifying when evidence becomes valid, how visual history is maintained, or how responses are triggered. As a result, similar scores may correspond to different workloads, failure modes, and operational behavior.
By Yibo Ma, Qianqian Zhang, Peng Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.