OSWorld-Pro: Process-based Evaluation for Computer Use Agents
OSWorld-Pro adds process-level subgoals so computer-use agent failures can be diagnosed beyond final outcomes.
OSWorld-Pro is a process-based benchmark for computer-use agents, with more than 300 tasks and 2,800 subgoals grounded in over 67,000 human annotations. Human-aligned LLM judges score subgoal completion so progress can be tracked across sequentially dependent steps, rather than only final deliverables. Claude Opus 5 reaches 75.7 percent on OSWorld-Pro versus 83.4 percent on OSWorld. The authors highlight failure modes such as subgoal-irrelevant actions and click-based GUI mistakes.
48