OverSeer adds fixed auxiliary views to multi-agent world models
Original title: OverSeer: Multi-Agent World Models Beyond What Agents See
A delta-VAE encodes only changes from a static reference, cutting tokens for auxiliary views.
OverSeer generates consistent agent-specific video by pairing egocentric views with spatially fixed auxiliary cameras that proxy a world-centric state. Experiments in Minecraft and Doom show better world and agent consistency, with qualitative results on real multi-camera captures. Compression analysis finds delta-VAE lowers the cost of generating those auxiliary views.
- Fixed views map changes to localized world-state updates.
- Auxiliary views are redundant and semi-static, so they compress well.
- Qualitative real-camera results suggest the method is not game-engine specific.
From the original article
1 min read at huggingface.co
Despite the success of video world models, extending them to multiple agents is difficult because it requires generating several agent-specific video streams that are consistent in time and between agents. To address this gap, we introduce OverSeer, a multi-agent world model that augments egocentric agent views with auxiliary camera views.
By Runjia Li, Kam Woh Ng, Minghao Chen · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.