Learning 3D Editing without Paired Supervision via Generative Prior Distillation
New framework distills 2D editing and VLM priors into a feed-forward 3D editing model without paired 3D training data.
The method, PriorEdit3D, learns feed-forward instruction-guided 3D editing by distilling knowledge from foundation models instead of using ground-truth 3D pairs. Through a differentiable rendering pipeline it supervises a 2D visual prior from an image editing model at the main view and a Vision-Language Model semantic prior at novel views for instruction fidelity and identity preservation. A 3D-aware Distribution Matching regularization constrains outputs to the manifold of realistic 3D assets defined by a pretrained image-to-3D teacher. Experiments report superior instruction fidelity and cross-view consistency over state-of-the-art baselines, with code released on GitHub.
- Eliminates need for paired 3D supervision via generative prior distillation
- Combines 2D image-editing prior and VLM semantic prior at novel views
- 3D-aware Distribution Matching regularization prevents geometric collapse and multi-view inconsistency
- Outperforms state-of-the-art 3D editing baselines in fidelity and cross-view consistency
- Tags
- 3d-editing3d-generationdifferentiable-renderingdistillationgenerative-priorsvision-language-model
- Products
- PriorEdit3D
From the original article
1 min read at huggingface.co
Instruction-guided 3D editing is essential for interactive content creation, yet it faces a significant bottleneck: the severe scarcity of high-quality paired training data. Existing approaches attempt to bypass this by either relying on slow test-time optimization or training on pseudo-pairs constructed via complex pipelines, which often introduce structural drift and geometric artifacts.
By Hao Wen, Weibin Yun, Hongxing Fan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.