ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs
ReactHuman benchmark tests whether multimodal LLMs react safely to sudden household hazards; seven evaluated models mishandle roughly one hazard in three.
ReactHuman is the first physics-grounded benchmark for human-like reactive decision-making, placing a multimodal LLM as the brain of a simulated humanoid facing 17 event families of sudden household hazards across over 1,000 bit-for-bit reproducible scenes with annotation-free ground truth from 240 Hz rigid-body simulation, including adversarial objects whose appearance contradicts their physics. A five-metric suite scores each reaction along reasonable, safe, and physically grounded axes, and every committed plan is physically executed. Seven representative MLLMs mishandle roughly one hazard in three, act from fixed dispositions rather than the observed scene, trust appearance over motion, and miss interception points at meter scale; none of these failures shrink with model scale.
- Over 1,000 reproducible scenes across 17 hazard event families with 240 Hz simulation ground truth
- Five metrics score reactions as reasonable, safe, and physically grounded
- Seven MLLMs mishandle roughly one hazard in three
- Failures do not shrink with model scale; models trust appearance over motion
- Includes adversarial objects whose looks contradict physics, like a foam anvil
From the original article
1 min read at huggingface.co
Reacting to sudden physical hazards (catching a slipping plate, dodging a falling knife) is both a meaningful test of embodied intelligence and a hard requirement for deploying multimodal large language models (MLLMs) as the decision coreof household robots.
By Yizhan Li, Jianxin You, Mengyang Xiong · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.