LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures
LMBuild benchmarks LLM agents on generating structures that are buildable, functional, and physically realizable.
LMBuild evaluates LLM agents on generating assembled structures that are buildable and functional, not merely geometrically plausible. It provides an interactive construction environment, CAD datasets augmented with Wikipedia knowledge, and metrics for structural soundness, functional affordance, design quality, and physical realization. Across 30 systems, frontier closed-source models no longer fail mainly on soundness, while affordance and operability remain harder, and functional specifications substantially improve part completeness and kinematics.
- Benchmark scores soundness, functional affordance, design quality, and physical realization
- Thirty systems tested; operability remains harder than geometric soundness
- Functional specifications improve completeness, kinematics, and physical operability
- Stronger models create new parts; weaker models rely more on retrieval
From the original article
1 min read at huggingface.co
LLM-based agents are increasingly capable of generating complex 3D structures, with the potential to reshape how objects are designed and realized in the physical world. Yet, producing elegant geometry is fundamentally different from producing objects that can be built and perform their intended functions. Existing evaluations largely focus on geometric quality while overlooking physical realizability.
By Jiateng Liu, Rushi Wang, Cheng Qian · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.