CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design
CADWorld benchmark evaluates computer-use agents on 200 long-horizon FreeCAD tasks; best agent reaches 17.5% success versus 87% expert reference.
CADWorld is a new benchmark for long-horizon computer use in FreeCAD, comprising 200 tasks across 11 mechanical-CAD workflow categories including sketching, part modeling, assembly, CAM, FEM, and technical drawing. Agents interact via screenshots and GUI actions, with success verified through executable checks on saved FreeCAD artifacts covering geometry, parametric structure, constraints, manufacturing state, and simulation results. Across seven current agents, the strongest achieved only 17.5% success versus an 87.0% expert reference pass, exposing a large gap between general GUI competence and reliable engineering workflow execution.
- 200 FreeCAD tasks across 11 mechanical-CAD workflow categories with executable artifact-based checks
- Best of seven evaluated agents achieves 17.5% success; expert reference passes 87.0%
- Weaker agents fail to produce valid artifacts; stronger agents fail on structural and geometric requirements
- Highlights gap between generic GUI competence and persistent, verifiable engineering workflows
Full article191 words · extracted from huggingface.co · click to collapse
Computer-use agents are increasingly evaluated in realistic desktop environments, but existing benchmarks provide limited coverage of professional engineering workflows whose outputs are persistent, structured artifacts. Mechanical computer-aided design (CAD) is a particularly demanding setting: an agent must manipulate geometry and constraints over long interaction horizons while producing a native project whose dimensions, construction structure, and downstream engineering state remain valid. We introduce CADWorld, a benchmark for long-horizon computer use in FreeCAD. CADWorld contains 200 tasks spanning 11 mechanical-CAD workflow categories, including sketching, part modeling, assembly, CAM, FEM, measurement, mesh processing, and technical drawing. Agents operate through screenshots and GUI actions, while success is determined by task-specific executable checks over saved FreeCAD artifacts and auxiliary outputs, covering geometric properties, parametric structure, constraints, manufacturing state, and simulation results. Across seven current agents on the full benchmark, the strongest agent achieves 17.5\% success, compared with an 87.0\% expert reference pass. We find that weaker agents often fail before producing a valid artifact, whereas stronger agents increasingly fail on structural, geometric, and construction-process requirements. CADWorld therefore exposes a gap between general GUI competence and reliable execution of persistent, verifiable engineering workflows. Project accessible at https://cad-world.github.io.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.16251