PAIR: Bridging Perception and Action in Vision-Language-Action Models
PAIR learns shared perception-action tokens that raise success for OpenVLA-OFT and VLA-Adapter robot policies.
PAIR is a training framework for continuous-action vision-language-action models that aligns visual-language features with expert action latents using a Masked Action Autoencoder and a Bridge Module. Bridge Tokens are injected as an action-ready starting point for Action Expert refinement; at inference the autoencoder is removed. On LIBERO-Plus, PAIR lifts VLA-Adapter success from 59.1% to 64.2%, and on CALVIN ABC-D average completed sequence length from 4.42 to 4.53. Across seven real-world tasks, OpenVLA-OFT success rises from 51.4% to 65.0%.
- Masked Action Autoencoder encodes expert action chunks into Action Latent Tokens
- Bridge Module aligns vision-language features with those action latents
- LIBERO-Plus: VLA-Adapter success rises from 59.1% to 64.2%
- Seven real-world tasks: OpenVLA-OFT success rises from 51.4% to 65.0%
- Autoencoder is removed at inference; tokens come from observation and instruction
From the original article
1 min read at huggingface.co
Vision-language-action (VLA) models map visual observations and language instructions to continuous robot actions. This task requires a transition from representations that describe the scene and instruction to representations that support action generation. Many continuous-action VLAs leave this transition implicit and supervise it mainly through the final action-prediction loss.
By Kaixi Feng, Guoheng Sun, Ang Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.