Same Trajectory, Contradictory Rewards (ROBORMBENCH): Paraphrase Fragility in Vision Language Reward Models
New ROBORMBENCH benchmark shows vision-language reward models can flip robot success/failure judgments when goal instructions are paraphrased.
The authors show that paraphrasing the instruction alone can substantially change progress scores from VLM reward models, even flipping identical robot trajectories between failure and success. ROBORMBENCH comprises 2,390 real-robot trajectories with ground-truth progress labels and 21,673 verified paraphrases covering lexical, syntactic, and action-goal rewrites. Instability is widespread across proprietary and open-source VLMs, grows with more divergent rewrites, and is not reliably reduced by scale or explicit reasoning, while trajectory-grounded dedicated reward models are markedly more stable.
35