CADWorld: Computer-Use Benchmark for Long-Horizon Computer-Aided Design
CADWorld benchmark evaluates computer-use agents on 200 long-horizon FreeCAD tasks; best agent reaches 17.5% success versus 87% expert reference.
CADWorld is a new benchmark for long-horizon computer use in FreeCAD, comprising 200 tasks across 11 mechanical-CAD workflow categories including sketching, part modeling, assembly, CAM, FEM, and technical drawing. Agents interact via screenshots and GUI actions, with success verified through executable checks on saved FreeCAD artifacts covering geometry, parametric structure, constraints, manufacturing state, and simulation results. Across seven current agents, the strongest achieved only 17.5% success versus an 87.0% expert reference pass, exposing a large gap between general GUI competence and reliable engineering workflow execution.
- 200 FreeCAD tasks across 11 mechanical-CAD workflow categories with executable artifact-based checks
- Best of seven evaluated agents achieves 17.5% success; expert reference passes 87.0%
- Weaker agents fail to produce valid artifacts; stronger agents fail on structural and geometric requirements
- Highlights gap between generic GUI competence and persistent, verifiable engineering workflows
From the original article
1 min read at huggingface.co
Computer-use agents are increasingly evaluated in realistic desktop environments, but existing benchmarks provide limited coverage of professional engineering workflows whose outputs are persistent, structured artifacts.
By Zihan Dong, Yuanzhe Liu, Zhiyuan Ma · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.