LEASH cuts harm rates in agentic robotics safety benchmark
Original title: LEASH: Stay Safe in Agentic Robotics
RoboRisk runs 92 simulated and 13 physical tasks; six foundation models generally showed high harm rates.
Researchers introduce RoboRisk, a closed-loop benchmark for safety when foundation models control robots, and LEASH, a harness with task, action, and commit gates. Six models, plus Dream API and PokeBot's RoboAPI, were evaluated. The authors report that LEASH reduced harm across all six models while largely preserving task completion and adding little time and token overhead.
Unconfirmed: Harm reductions and overhead figures are the authors' experimental results, not independently replicated.
- Tasks cover refusal, plan repair, execution hazards, injection, and sequential memory.
- Gates use Jev or OpenAI's Decisions API and share one safety policy with the planner.
- Tests ran in simulation and on physical robots.
- Authors say prior benchmarks cover at most two of the five categories.
- Organizations
- OpenAI
From the original article
1 min read at huggingface.co
Foundation models such as GPT and Claude have shown strong capabilities in spatial understanding and embodied intelligence. These advances make agentic robotics a promising path toward artificial general intelligence in robotics. However, we find that using foundation models to control robots poses serious safety risks, and comprehensive safety evaluation is still lacking.
By Qinfeng Zhu, Weiguang Zhao, Ke Sun · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.