ASTEROID distillation teaches in-context exploration under partial observability
Original title: Asymmetric On-Policy Distillation Learns In-Context Exploration
Across 11 environments it reports about 2x returns, about 100x fewer interactions than RL, and +40% real-robot success.
ASTEROID is a student-teacher supervised framework for learning test-time exploration when the agent cannot see the full state. A history-conditioned student gathers on-policy context, and a privileged expert labels actions on that context, so the student searches while uncertain and exploits once the hidden state is known. The authors report results on eleven environments spanning discrete benchmarks, manipulation, navigation, and Procgen, plus transfer from simulation to vision-denied real-world robot manipulation. They say it beats frontier models at exploration and uses far fewer interactions than reinforcement-learning baselines.
Unconfirmed: Performance figures are the authors' reported comparisons, not an independent evaluation.
- Student collects on-policy partial-observability context; expert labels actions on that history.
- Reported ~2x higher returns under fixed exploration budgets versus baselines.
- About 100x fewer interactions than the RL baselines they compare.
- Sim-to-real vision-denied manipulation success rises +40% over standard distillation.
From the original article
1 min read at arxiv.orgSequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to learn exploration.
By Sriyash Poddar · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.