BrickBench: Evaluating Agentic Brick Design

Source facts

The preprint introduces BrickBench, a benchmark for text-conditioned LEGO-set design. Agents must produce assemblies that satisfy semantic and design criteria while remaining physically buildable, selecting parts from a discrete library and reasoning over local and global constraints. The benchmark scores validity, alignment, and design across three settings. The authors also release BrickAgent, an environment for coding agents to construct, inspect, and validate designs, and report that leading agents remain below human designs.

Analysis

BrickBench couples semantic intent with a physical construction constraint, making it a compact test of whether an agent can turn a language-level plan into a realizable object. The gap from verifiable requirements to human-quality design leaves open questions about search, creativity, and generalization beyond the part library.

Uncertainty/Doubt

• This is an arXiv preprint. The benchmark release, agent comparisons, and human-design gap were not independently reproduced here.