SurgStream proposes proactive streaming framework for surgical video understanding
Original title: SurgStream: A Workflow-Anchored Proactive Streaming Framework and Benchmark for Real-Time Surgical Video Understanding
SurgStream with Workflow-Anchored Memory and Deadline-aware Transition Learning provides SurgStream-Bench of 11.7K interactions for real-time surgical video tasks.
SurgStream is a workflow-anchored proactive streaming framework for real-time surgical video understanding. Workflow-Anchored Memory records structured instrument-action-target observations and folds them into phase-level summaries. Deadline-aware Transition Learning supervises response states within clinical windows. Experiments on SurgStream-Bench show gains over state-of-the-art especially in memory and forward proactive QA.
- SurgStream-Bench comprises 11.7K streaming interactions
- Tasks include real-time perception, backward memory queries, forward proactive QA, proactive alerts and caption memory
- WAM provides compact queryable surgical memory
- DTL enables timely responses before procedural deadlines
From the original article
1 min read at arxiv.orgSurgical video understanding is essential for intraoperative assistance such as workflow monitoring, instrument preparation and procedural documentation. Delivering such assistance in real time requires a model to jointly perceive the ongoing scene, remember past surgical events, and respond before clinically relevant moments pass.
By Ruipeng Ma · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.