Memorizon: Training World Models Beyond Their Context Window
Memorizon trains world models on long revisits using a bounded retrieved latent bank instead of full-span attention.
Memorizon trains streaming world models for consistent revisits by scoring only the last k chunks of arbitrarily long samples and retrieving a bounded bank of camera-covisible latents instead of tokenizing the full history. Extending the supervised span from 100 to 400 seconds adds about 12% step time, and spans that reach the first visit improve revisit consistency by 24% to 30% over a sliding-window baseline, with some image-quality cost. Filling the bank from another episode cuts revisit correlation by 83%, showing the model uses retrieved content.
- Memorizon supervises long revisit spans without attending over every intervening frame.
- Shared latent bank is bounded by k times K retrieved camera-covisible latents.
- Extending span from 100 to 400 seconds adds 12% step time.
- Longer spans add 24% to 30% revisit consistency versus a sliding window.
From the original article
1 min read at huggingface.co
Streaming world models should render a place consistently across repeated visits. Directly supervising such revisits requires training samples that capture both visits, often spanning minutes. Yet dense attention over the full span incurs quadratic costs, making long-span supervision expensive.
By Tingting Liao, Xuezhi Liang, Hao Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.