SO-101 Bench benchmarks language-conditioned spatial robot manipulation
Original title: SO-101 Bench: Accessible Spatial Manipulation
A simulated twin matched the real arm's policy ranking exactly, though simulation overestimated success.
The paper introduces SO-101 Bench, a simulated benchmark for language-conditioned spatial manipulation on an Isaac Lab digital twin of a real SO-101 arm. It includes four tasks scored as five families, 138 objects, 916 evaluation episodes, and 3,629 simulated demonstrations used to fine-tune policies under an equal compute budget. Five policies trained without real data kept the same ranking on the physical arm in an unseen environment (Spearman rho = 1.00), although simulation overestimated success. The best learned policies scored near 61 percent, while an LLM agent that writes code and delegates grasping scored 83.2 percent.
- Object splits cover seen, unseen-instance, and unseen-class cases.
- Subsets test held-out appearance, rotate-to-fit, and occlusion memory.
- Nine VLAs and world-action models were compared.
- An ablation attributes the LLM agent's lead to harness and model choice.
- Spatial placement, unseen classes, and rotate-to-fit remain open.
- Products
- SO-101SO-101 BenchIsaac Lab
From the original article
1 min read at huggingface.co
Low-cost arms such as the SO-101 have become the most common entry point to robot learning, yet no released policy performs spatially constrained, language-conditioned tasks on them out of the box, and existing SO-101 benchmarks run only on physical hardware with simple tasks and few held-out objects.
By Truman Hickok · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.