PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation
PINT coordinates vision-language, video, and action models via shared Predictive Intent for stronger robot manipulation.
PINT is a unified robot-manipulation model that shares chunk-level Predictive Intent across a vision-language model, a video generation model, and an action expert while keeping expert-specific pathways. Intent Tokens condition the video model and action expert; the action expert reads video representations through one-way attention at inference. Without embodied pretraining, it reports 15.5% on RoboTwin 2.0-Hard versus 1.8% for Fast-WAM and 3.2% for StarVLA, and 79.5% on LIBERO-Plus versus 51.5% and 75.0%. On five real bimanual tasks it reaches 68% in-distribution and 24% out-of-distribution success, versus 35%/9% for pretrained Motus and 67%/35% for pretrained pi0.5.
- Learnable Intent Tokens coordinate a VLM, video model, and action expert.
- Two-stage training freezes the video model, then adapts the VGM and action expert.
- RoboTwin 2.0-Hard: 15.5% versus 1.8% Fast-WAM and 3.2% StarVLA.
- LIBERO-Plus: 79.5% versus 51.5% and 75.0%.
- Five real bimanual tasks: 68% ID and 24% OOD success.
- Products
- RoboTwin 2.0LIBERO-Plus
From the original article
1 min read at huggingface.co
Generalist robot manipulation can benefit from pretrained vision-language models (VLMs) for semantic reasoning and video generation models (VGMs) for world modeling. However, existing unified models that directly fuse them through shared attention at multiple layers can make it harder to draw on their specialized capabilities.
By Xinyu Guo, Chao Jing, Li Yi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.