BrickBench: Evaluating Agentic Brick Design
BrickBench scores agents on designing physically buildable LEGO assemblies from a discrete parts library.
BrickBench is a benchmark for text-conditioned agents that design LEGO sets which must be physically buildable, not only semantically plausible. Agents select parts from a discrete library and reason about local and global constraints, with scores for validity, alignment, and design across three settings that vary scale and part availability. The authors provide BrickAgent, an environment for coding agents to construct, inspect, and validate designs. Leading agents largely satisfy verifiable physical and semantic requirements but fall short of human designs.
- Agents select discrete parts under local and global build constraints.
- Scores cover validity, alignment, and design in three settings.
- BrickAgent lets coding agents construct, inspect, and validate designs.
- Leading agents meet checks but still fall short of human designs.
From the original article
1 min read at arxiv.orgWe propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints.
By Peter Kulits · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.