NestedVLA: Learning to Consolidate and Generate Skills for Vision-Language-Action Models
NestedVLA generates task-specific VLA parameters, beating model merging by 41 points across three robotics benchmarks.
NestedVLA consolidates robotic skills by generating task-conditioned action-expert parameters with a nested hypernetwork instead of merging weights. Those parameters compose with a frozen base vision-language-action model, while compressed task-specific VLM residuals stay separate. Across MetaWorld, ManiSkill, and CALVIN, it outperforms the strongest model-merging baseline by 41 percentage points on average, matches per-task fine-tuning within 1 point on ManiSkill, and narrows the CALVIN gap.
- NestedVLA generates task-adaptive action-expert parameters with a hypernetwork.
- A frozen base VLA is combined with separately stored compressed residuals.
- It beats the strongest merging baseline by 41 percentage points on average.
- Results span MetaWorld, ManiSkill, and CALVIN.
- On ManiSkill it matches per-task fine-tuning within 1 percentage point.
From the original article
1 min read at arxiv.orgVision--Language--Action (VLA) models promise generalist robotic agents, yet consolidating skills learned across heterogeneous tasks and environments remains difficult: training data is fragmented across platforms, joint multi-task training suffers from interference, and model merging tends to introduce parameter conflicts or require architectural changes.
By Guodong Du · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.