DeformSmith: Physics Harness-Guided Hierarchical Generation of Deformable Assets for Robot Manipulation
DeformSmith generates physically credible deformable assets for robot manipulation from text or a single image using hierarchical agentic construction and physics-guided testing.
DeformSmith progressively builds, tests, and refines geometry, physical models, material behavior, and robot interaction through a shared physics-grounded harness until assets are ready for simulation. Robot manipulation feedback and replayable interaction data close the generation loop. It outperforms PhysGen3D, PhysGM, and PhysX-Omni on visual quality and physical plausibility while supporting data synthesis for deformable-object manipulation.
- Generates interactive deformable assets from text or single image
- Shared physics harness iteratively builds, tests, and refines assets
- Outperforms PhysGen3D, PhysGM, and PhysX-Omni baselines
Full article167 words · extracted from huggingface.co · click to collapse
Creating deformable assets for robot manipulation requires jointly specifying their geometry, appearance, and physical properties. This is especially challenging for deformable objects, since text and images provide limited evidence about how they deform and respond to contact, yet these responses directly affect their suitability for interaction. Automated generation therefore needs to resolve coupled physical requirements and use interaction evidence to guide construction and refinement. We present DeformSmith, a framework that enables automated generation of interactive, physically credible deformable assets from text or a single image. Through hierarchical agentic construction and a shared physics-grounded harness, it progressively builds, tests, and refines geometry, physical models, material behavior, and robot interaction until the resulting asset is ready for simulation and manipulation. Robot interaction closes the generation loop through manipulation feedback and replayable interaction data. Results show that DeformSmith generates assets with better visual quality and physical plausibility than state-of-the-art baselines, including PhysGen3D, PhysGM, and PhysX-Omni, while supporting the synthesis of data for robotic manipulation of deformable objects. Project page: https://can-lee.github.io/deformsmith-web/
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.18620