HeLM Improves Non-Markovian Manipulation in VLA Models
Original title: HeLM: Hierarchical Explicit Language Memory for Non-Markovian Manipulation
HeLM achieves 12.8 percentage point intra-episode success gain on 10 non-Markovian tasks with 56 variants.
Vision-Language-Action models assume Markovian transitions and struggle with persistent context needs. HeLM introduces hierarchical explicit language memory using factorized event detection and task-triggered updates. Evaluation on real-world tasks shows improved success rates, compositional generalization and efficiency over keyframe baselines.
- 12.8 percentage points improvement in intra-episode success
- 78% subtask success on compositional generalization
- Up to 4.5x less memory and 2.5x faster at long horizons
- Evaluated on 10 tasks with 56 variants including inter-episode tasks
From the original article
1 min read at huggingface.co
Recent advances in Vision-Language-Action (VLA) models have enabled fine manipulation capabilities, yet performing tasks that require working memory remains a key challenge. Most existing VLA architectures operate under a Markovian assumption, mapping only the current observation to an action, which leads to failure in non-Markovian environments.
By Gyeonghun Kim, Donghoon Kim, Minsoo Choi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.