EngiWorld: What Can Frontier Agents Deliver in Professional Engineering Environments?
EngiWorld benchmarks frontier agents on 1,301 professional engineering tasks; the best EngiScore is only 44.3.
EngiWorld is a benchmark of 1,301 expert-curated tasks across CAD, CAE, CAM, BIM, EDA, and 3D visualization on 26 professional software platforms, with GUI and CLI interfaces and six task types. An artifact-centric verifier checks geometric validity, physical feasibility, and rule compliance, scoring quantitative designs by specification attainment rather than binary success. Among seven frontier models, the strongest reached an EngiScore of only 44.3, and just 3.6% of multi-software attempts succeeded.
- 1,301 tasks cover six domains and 26 software platforms.
- Checks geometry, physical feasibility, and rule compliance.
- Best of seven frontier models scored EngiScore 44.3.
- Only 3.6% of multi-software attempts succeeded.
From the original article
1 min read at huggingface.co
Autonomous agents have made rapid progress in general-purpose computer use, but reliable automation of professional industrial engineering remains out of reach, as engineering workflows demand reasoning over geometric and physical constraints and dependencies preserved across software and design stages.
By Hongcheng Gao, Hailong Qu, Yu Lei · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.