OSMA-Bench finds robot policies fail obstacle-conditioned safe motion
Original title: Benchmarking Obstacle-Conditioned Safety in Household Robot Navigation
VLM and VLA policies can rank obstacles by safety but do not change clearance, speed, or smoothness to match.
Researchers introduce OSMA-Bench, which uses spatially matched household scenarios that differ only by obstacle type, and the Semantic Sensitivity Index to test whether clearance, speed, and smoothness follow a caution ordering. Evaluated VLM-based and vision-language-action policies did not consistently adapt motion to obstacle safety, even with explicit safety prompts. The paper says they can distinguish obstacles by safety relevance but fail to reflect that in motion parameters. For VLM policies, an explicit parameter-inference step between recognition and program generation improved adaptation.
Unconfirmed: Findings are the authors' evaluation of unnamed VLM and VLA policies, not independently verified.
- Scenarios are spatially matched and differ only in obstacle type.
- SSI checks clearance and distance-dependent speed and smoothness.
- Explicit safety prompts did not produce consistent motion adaptation.
- A parameter-inference step improved VLM-based policy adaptation.
From the original article
1 min read at huggingface.co
Safe household navigation requires robots to regulate clearance and speed according to what an obstacle is and how close it is, without unnecessary conservatism. Yet existing evaluations do not test whether semantic safety distinctions are reflected in continuous motion.
By Hyunwoong Bae, Jisu Han, Jiyeon Choi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.