UniWALM unifies video, action and language for robot control
Original title: UniWALM: Unified World Action Language Model for Embodied Robot Control
Authors report 34.5% on RMBench memory tasks without a pretrained language model, and Find-the-Item success rising from 11.1% to 61.1%.
UniWALM learns video, action, and language in one mixture-of-transformers backbone whose diffusion heads co-denoise, so generated language is reused as memory without a separate planner. Flow Matching with Correction trains recovery from perturbed action trajectories at no inference cost. The paper reports gains over same-backbone baselines on MolmoSpaces pick-and-place and the memory-dependent subset of RMBench, and over compared VLAs on a real Find-the-Item task.
Unconfirmed: Scores are the authors' reported comparisons; independent replication is not stated.
- Language and action are decoded in the same pass, with no external memory bank.
- RMBench memory subset score is 34.5%, above Mem-0's 28.5% keyframe setup.
- Find-the-Item success rises from 11.1% to 61.1% when textual memory is generated.
- Compared state-of-the-art VLAs stay at or below 33.3% on that task.
From the original article
1 min read at arxiv.orgMotivated by the human ability to integrate experience across language, perception, and action, we introduce UniWALM, a unified world-action-language model that learns video, action, and language in a single shared representation, where each modality is both a conditioning signal and a supervision target.
By Rahul Sajnani · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.