ExRL adapts long-chunk robot policies by learning execution length
Original title: ExRL: Sample-Efficient Online Adaptation of Long-Chunk Policies via Execution Length Control
A lightweight off-policy RL adapter predicts how long to run each action chunk, cutting the adaptation space versus optimizing the full chunk.
The paper argues that reinforcement learning over long action chunks needs many online samples because the optimized action space is high-dimensional, which is costly on real robots. Execution at the Right Length (ExRL) instead trains a lightweight adapter to predict how long to execute each predicted chunk, using off-policy RL while leaving the base policy in place. The authors report sample-efficient adaptation across simulation benchmarks and real-world robotic tasks with little task-specific hyperparameter tuning.
- Adaptation shifts from action optimization to execution-length prediction.
- The adapter is trained with off-policy reinforcement learning.
- Expressivity of the low-dimensional adapter grows with chunk size.
- Evaluated on simulation benchmarks and real-world robotic tasks.
From the original article
1 min read at huggingface.co
Predicting long action chunks, often spanning dozens of timesteps, has become a key ingredient in increasingly capable robot policies.
By Xu Luo, Jiaying Yang, Hao Wu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.