Visual memory raises recall but lowers precision in long video
Original title: More Memory, Better Videos? A Controlled Study of Visual Memory in Long-Video Generation
On VMem-Bench, retrieval lifted entity recall from 0.559 to 0.628-0.693 while precision and forbidden-entity avoidance fell.
Long-video generators retrieve past frames to limit identity drift, but an image faithful to the past can be wrong for the present story. VMem-Bench measures reference selection on 92 real films before generation, then compares four retrieval policies with no-memory and gold-informed controls on 30 generated stories using one shared generator. Retrieval raised entity recall and similarity to past appearances, but lowered precision and forbidden-entity avoidance and can reinforce obsolete looks after the story changes. High long-gap recall can also occur without memory when the prompt itself supplies the appearance.
Unconfirmed: Effects are reported for this shared generator and VMem-Bench, not as a general ranking of memory systems.
- Track-A used 92 films and 16,548 segments before generation.
- Track-B used 30 stories and 2,824 segments with one generator.
- Entity recall rose from 0.559 to 0.628-0.693 with retrieval.
- Precision fell from 0.977 to 0.915-0.938.
- Forbidden-entity avoidance fell from 0.968 to 0.845-0.898.
From the original article
1 min read at huggingface.co
Long-video generators increasingly retrieve past visual observations to prevent identity drift. The intuition is compelling: more access to history should make future video more consistent. But an image can be faithful to the past and wrong for the present.
By Yuzhuo Chen, Huafeng Shi, Xinyu Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.