BrickBench: Evaluating Agentic Brick Design
BrickBench scores coding agents on designing text-conditioned LEGO sets that can actually be built.
BrickBench is a benchmark for text-conditioned agentic LEGO-set design, requiring an agent to select parts from a discrete library and produce an assembly that is semantically appropriate and physically buildable. It scores validity, alignment, and design across three settings that vary in scale and part availability, and ships BrickAgent so coding agents can construct, inspect, and validate designs. Leading agents largely satisfy verifiable physical and semantic requirements but fall short of human designs. The benchmark and environment are released at brickben.ch.
- Agents must pick discrete LEGO parts and satisfy local and global build constraints.
- Scoring covers validity, alignment, and design across three scale and inventory settings.
- BrickAgent lets coding agents construct, inspect, and validate assemblies.
- Leading agents meet physical and semantic checks but lag human designs.
From the original article
1 min read at huggingface.co
We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints.
By Peter Kulits, Yiqing Xu, R. Kenny Jones · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.