GoDeep: Annotation-Free Open-Vocabulary 3D Scene Understanding via Language-Space Lifting
GoDeep achieves annotation-free open-vocabulary 3D segmentation by grounding structured image descriptions in language-only embeddings, outperforming CLIP-based lifting on out-of-vocabulary objects.
GoDeep uses a vision-language model purely as a translator, producing structured entity-level image descriptions that are grounded, projected, and aggregated in a general-purpose language-only embedding space, with no 3D training corpus or dedicated 3D encoder required. On ScanNet++ the pipeline is competitive with strong annotation-free baselines, and on a cultural-heritage benchmark a systematic vocabulary correction reverses initial CLIP-based rankings. Language-space embeddings separate genuinely out-of-vocabulary objects more sharply, localize them within scenes, and keep all predictions explainable as discrete text.