Ambient @ EgoProactive 2026 : Proactive Egocentric Assistance with Visually Grounded Supervision
ECCV 2026 challenge winner reformulates egocentric intervention timing as single-token classification, boosting macro-F1 by 0.249 over free-form generation.
The paper describes the winning submission to the EgoProactive track of the ECCV 2026 Wearable AI Challenge, ranking first in the large-model division and second in the <=2B division. The method reformulates intervention timing as single-token yes/no classification, improving macro-F1 by 0.249 and G-mean by 0.30 over free-form generation. Supervision generated by a tool-calling video agent transferred better than a narration-only dataset that was four times larger and ten times cheaper, suggesting visual grounding matters more than annotation volume.
- Ranked first in large-model division at EgoProactive 2026
- Single-token classification improved macro-F1 by 0.249
- Video-agent supervision outperformed larger, cheaper narration-only data
Full article164 words · extracted from huggingface.co · click to collapse
We present our submission to the EgoProactive track of the ECCV 2026 Wearable AI Challenge, which ranked first in the large-model division and second in the <=2B division. The task requires a wearable assistant to decide after each eight-second segment of egocentric video whether to intervene or remain silent. Our approach has two main components. First, we reformulate intervention timing as single-token classification. Rather than generating either interrupt<utterance> or silent, the model predicts yes or no, and we derive the decision from the renormalised probabilities of these two tokens. This formulation improved macro-F1 by 0.249 and G-mean by 0.30 over free-form generation. Second, because labelled data were limited to the released validation set, we generated additional supervision using a tool-calling video agent that inspects each clip and assigns intervention timestamps. A narration-only alternative was four times larger and ten times cheaper, but transferred worse than supervision from an unrelated real corpus, suggesting that visual grounding is more important than annotation volume for this task.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.07099