Workspace Models: Lightweight Robotic Memory via Saliency-Driven Supervision
Researchers propose workspace tokens, a lightweight latent robot memory distilled from train-time VLM saliency queries, improving policy performance without in-loop VLM calls.
The paper introduces workspace tokens, a latent memory representation for robotic manipulation policies trained by using a VLM to identify task-relevant current and historical information and distilling it via a set-reconstruction decoder loss. At deployment the workspace token serves as a drop-in replacement for observations, letting policies solve memory-intensive tasks without expensive in-the-loop VLM queries. Experiments in simulation and on hardware show it is both more lightweight and yields better policy performance than conditioning on full histories or in-loop VLM compression.