Never Look Back: Understanding Persistence in 3D Object Memory from Egocentric Videos
Ledger, a persistent 3D object memory, lifts HD-EPIC accuracy from 29.7% to 42.6% on egocentric video.
Researchers present Ledger, a persistent 3D object memory built from egocentric video that stores locations, histories, and short contextual descriptions after objects leave view. It clusters observations by resting location and records a move only after repeated evidence, reducing localization noise. Ledger raises HD-EPIC accuracy from 29.7% to 42.6% and UCS-Bench from 33.8% to 38.5%, and localizes Ego4D objects with a 0.99 meter median error on returned predictions. Tests on 100 multi-scene streams still show retrieval and construction failures.
- Ledger keeps object locations, histories, and short descriptions.
- HD-EPIC accuracy rose from 29.7% to 42.6%.
- UCS-Bench accuracy rose from 33.8% to 38.5%.
- Ego4D localization median error was 0.99 meters.
From the original article
1 min read at arxiv.orgAs we move through the world and carry out everyday tasks, we encounter objects that may become relevant only later. We are capable of recalling where we left something or what was inside a container, even without knowing we would need it later. Here, we study how an embodied assistant can build a similar memory from egocentric videos, by observing a person's day-to-day activities.
By Shravan Chaudhari · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.