Geometric and Semantic Coupling for Interaction Understanding in 3D Scenes
Segment-Snap combines geometric motion priors and semantic part-handle predictors for 3D interaction understanding.
Segment-Snap couples geometric and semantic cues to describe movable parts, handles, and their motion in 3D scenes. A geometric decoder uses planar and upright priors and predicted handle locations to choose hinge lines, while a joint predictor adds handle candidates and part-based class corrections. On Articulate3D validation, handle guidance raises motion-gated average precision from 13.74% to 40.98% at fixed masks and axes, and full context raises handle AP to 30.99%.
- Segment-Snap links movable parts, handles, and motion without a learned motion regressor.
- Handle guidance raises motion-gated AP from 13.74% to 40.98% on Articulate3D.
- Extra handle candidates lift handle AP from 24.63% to 29.65%; full context reaches 30.99%.
- Each geometric and semantic transfer is applied once, with no iterative feedback.
- Products
- Articulate3D
- AI models
- Segment-Snap
From the original article
1 min read at huggingface.co
Interaction understanding in 3D scenes requires a joint description of movable parts, their motion, and the regions through which they can be operated. We present Segment-Snap, which connects these outputs through the physical relationship between parts and handles. Learned predictors identify broad part surfaces and small handles.
By Hanyang Kong, Xingyi Yang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.