Just Action Models match world-action policies without visual generation
Original title: Vision-Language-Action or World-Action? Just Action Models
Against FastWAM, reported LIBERO success moves from 97.0%, 55.3%, and 91.2% to 98.2%, 64.5%, and 91.4%.
The paper proposes Just Action Model (JAM), a pretrained visual backbone plus an action expert trained with an action loss and a foresight loss. A prediction head used only in training is discarded, so neither training nor inference generates images. Authors report that JAM beats matched FastWAM and ImageWAM policies on LIBERO and LIBERO-Plus and is similar or slightly higher on RoboTwin, while cutting estimated online training FLOPs per sample per update by 73% versus FastWAM when foresight targets are cached.
Unconfirmed: Success rates and the FLOPs reduction are the authors' estimates, not an independent evaluation.
- The framework separates backbone choice, the future-observation objective, and inference machinery.
- Foresight targets are frozen encodings of recorded future observations.
- Either generator-initialized or vision-language backbones can be used.
- No extra visual-generation step runs at inference.
- Products
- LIBEROLIBERO-PlusRoboTwin
- AI models
- Just Action ModelFastWAMImageWAM
From the original article
1 min read at huggingface.co
The vision-language-action (VLA) and world-action model (WAM) communities debate whether control policies should inherit pretrained knowledge from vision-language models or from visual generators. The debate, however, bundles three separable choices: backbone initialization, the objective applied to future observations, and whether the machinery for that future loss is kept at inference.
By Yonghao Yu, Lang Huang, Runyi Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.