Beyond the Timeline: Augmenting Long-Video Memory with Grounded Entity Biographies
Grounded Entity Biographies lift long-video QA to 72.0% on EgoLifeQA.
Grounded Entity Biographies (GEB) is a long-video memory framework that groups visually grounded observations of the same physical instance into retrievable biographies while keeping each moment's context. At question time the biography is retrieved with episodic evidence so the model can follow an entity using identity links built during memory construction. Across four benchmarks, including day-long and week-long recordings, GEB improves both multiple-choice and open-ended question answering over prior memory frameworks. On EgoLifeQA it reaches 72.0% accuracy, 4.4 percentage points above the best published result; ablations show extra descriptions alone do not recover the gains.
- GEB links visually grounded observations of the same physical object.
- Biographies are retrieved with episodic evidence during question answering.
- Tested on four benchmarks including day-long and week-long video.
- EgoLifeQA accuracy reaches 72.0%, 4.4 points above the prior best.
- Ablations credit identity association and biography reading for the gains.
From the original article
1 min read at arxiv.orgAnswering questions about long videos often requires connecting events involving the same objects across hours or days. Chronological descriptions and text-derived entities can leave physical identity unresolved: different objects may share a description, while observations of the same object remain disconnected across events.
By Hui Ren · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.