The Past Frames the Future: Memory for Autoregressive Video Generation
A survey frames memory for autoregressive video generation as history beyond the context window.
This survey reviews memory mechanisms for autoregressive video generation, where bounded context, storage, and compute drop entity identities, dynamic states, and intervention-caused changes before they stop mattering. It defines memory as persistent historical information that can still influence later generation after the original evidence leaves the active context. The literature is organized into forms, functions, operations, learning, and evaluation. Open challenges include composable resource-aware architectures, trustworthy state updates, self-rollout learning, and standardized evaluation.
- Defines memory as history that persists beyond the context window
- Organizes work by forms, functions, operations, learning, and evaluation
- Notes loss of identity, dynamic state, and causal changes
- Flags composable memory, trustworthy updates, and standard tests
Full article230 words · extracted from huggingface.co · click to collapse
Advances in generative models have improved video fidelity, enabling long-horizon generation, interactive world modeling, and evolving visual environments. Autoregressive (AR) video generation extends visual sequences through causal rollouts. However, a fundamental bottleneck emerges: as the generated sequence expands, practical models must operate under strictly bounded context windows, storage, and computational limits. Consequently, critical historical information, e.g., entity identities, dynamic states, and intervention-induced causal changes, often leaves the active context long before its relevance diminishes. Overcoming this limitation and maintaining temporal persistence constitutes a fundamental memory problem. We present a systematic and comprehensive review of memory mechanisms in AR video generation. We formulate memory operationally as persistent historical information maintained across outer AR steps, capable of influencing future generation even after the originating evidence is no longer locally accessible. Building upon this unified framework, we organize the literature through five complementary perspectives: (I) Forms, the representational carriers of history; (II) Functions, the specific semantic and physical information requiring preservation; (III) Operations, the lifecycle of writing, reading, updating, managing, and integrating memory; (IV) Learning, the optimization of memory behaviors under closed-loop rollouts; and (V) Evaluation, the paradigms for diagnosing genuine memory capabilities. We conclude by synthesizing open challenges, including composable and resource-aware memory architectures, trustworthy state updating, self-rollout learning, and standardized evaluation. By bridging representations, mechanisms, and learning paradigms, this paper establishes a structured foundation for developing reliable, memory-conditioned video generation systems.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.28466