TouchScale: 500 Hours of Human Vision and Touch for Visual-Tactile Learning
TouchScale's 500 hours of vision and touch raise robot task success from 22.5% to 57.5%.
TouchScale is a 500-hour dataset of contact-rich human interaction recorded with a single wearable setup, aligning egocentric RGB-D video, wrist RGB video, and dense bimanual tactile measurements across roughly 2,000 tasks. Full-dataset training raised zero-shot contact IoU on an unseen tactile sensor from 0.134 to 0.383. Using it for visual-tactile mid-training improved average real-world robot success across four manipulation tasks from 22.5% to 57.5%. Performance generally increased as more consistently collected data was added, and a public release is planned.
- TouchScale contains 500 hours from one unified wearable setup.
- About 2,000 tasks span everyday activity and structured manipulation.
- Unseen-sensor contact IoU rose from 0.134 to 0.383.
- Robot success on four tasks rose from 22.5% to 57.5%.
- Recordings and reconstructed object models will be released publicly.
From the original article
1 min read at huggingface.co
Large-scale egocentric human interaction data is becoming an important source of physical supervision for embodied learning, yet video alone leaves the contact and pressure that characterize physical interaction unrecorded. Recent visual-tactile datasets provide this missing supervision, but their synchronized tactile data remain far smaller in volume than human video.
By Dayou Li, Hao Wang, Qianqian Yang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.