3DCodeEdit benchmark: models average 37% on 3D code edits
Original title: 3DCodeEdit: Benchmarking Instruction-Guided 3D Program Editing
Reviewer-model feedback on renders helps more than extra reasoning or an agent harness, and can lift a 27B model past a 2.8T model's single-shot rate.
3DCodeEdit evaluates instruction-guided editing of programs that build 3D objects in Blender, rather than generation from scratch. Tasks cover parameter changes, structural edits, style redesign, and scene composition, scored from renders against human-reviewed checklists. Across 17 models the average success rate is 37%, and the strongest models finish fewer than 60% of structural and style edits. Failures increasingly produce runnable code that builds the wrong geometry.
- The benchmark has 2,592 tasks over 108 Blender programs.
- Judging uses rendered outputs and human-reviewed checklists.
- Stronger models fail more often on geometric foresight than on execution.
- A separate reviewer describing render mismatches is the most helpful feedback.
From the original article
1 min read at arxiv.orgLanguage models increasingly create 3D content through code, with practical workflows often relying on editing existing programs rather than regenerating them, for speed and precise control. Such edits require understanding what a program builds and foreseeing how a change reshapes it, yet prior benchmarks mostly evaluate generation or simple edits against a reference.
By Binjie Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.