Cleanly-DINO learns visual task spaces for robotic cleaning
Original title: Cleanly-DINO: Efficiently Learning Visual Task Spaces for Robotics
A projection over frozen ViT features localizes unfinished regions and served as a reward in real wiping.
Cleanly-DINO learns a visual task space for tasks defined by appearance change rather than object pose. Given a mask of a task-relevant region and frozen vision-transformer features, it fits a projection and a progression metric from few demonstrations. The authors train on diverse cleaning examples and test localization of incomplete regions and transfer across visual domains. They also use the model as an observation encoder and reward in reinforcement learning and in a real robotic wiping pipeline.
- The method isolates a task space from a mask and frozen ViT features.
- Evaluation covers incomplete-region localization and transfer across visual domains.
- A fitted model guided execution inside a real-world robotic wiping pipeline.
From the original article
1 min read at arxiv.orgLearning manipulation tasks from few demonstrations requires abstractions that capture the changes relevant to task completion, such as object poses in rigid-body manipulation. However, tasks such as cleaning are defined by changes in visual state rather than object pose, making the choice of task representation less clear. In this work, we propose Cleanly-DINO to learn visual task spaces from few demonstrations.
By Adrian Röfer · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.