HARMONY: Hierarchical Agentic Reasoning for MONocular Image-to-Scene Synthesis
HARMONY reconstructs indoor 3D scenes from one image using hierarchical agentic reasoning and geometry refinement.
HARMONY reconstructs a complete indoor 3D scene from a single monocular image by combining agentic vision-language reasoning with visual geometry models. Starting from an empty floorplan, it calibrates the camera, recovers room layout and an initial placement order, then inserts wall-mounted elements, free-standing furniture, and dependent decorations using depth-first traversal and a feedback loop. After each vision-language placement, point-cloud estimates refine geometry so the render better matches the input. Experiments on synthetic and real images outperform the evaluated reconstruction baselines, and qualitative comparisons with GPT-6 Astra show more faithful object arrangements and scene detail.
- Hierarchical agentic reasoning reconstructs indoor scenes from one image.
- Objects are placed from walls and furniture down to decorations.
- Point-cloud refinement aligns each placement with the reference photo.
- Qualitative results are compared with GPT-6 Astra.
Full article257 words · extracted from huggingface.co · click to collapse
Compositional 3D scene reconstruction has recently been explored from two directions: agentic reasoning that provides semantic understanding of spatial relationships but lacks precise alignment with input images; and visual geometry foundation models that predict dense point maps from input images but the reconstruction quality is limited. Therefore, recovering a complete 3D scene from a single monocular image with accurate inter-object relationships and high-fidelity reconstruction quality remains challenging. In this paper, we present HARMONY, a hierarchical chain-of-thought framework that leverages both agentic reasoning and visual geometry foundation. Given an image of an indoor scene, starting from an empty 3D floorplan, HARMONY first calibrates the camera against the reference image to establish a semantically-grounded spatial frame, then uses agentic VLM reasoning to recover the 3D room layout and an initial placement order. It then places the objects in a hierarchical order, from wall-mounted elements, free-standing furniture, to dependent decorations on top of furniture. We also use depth-first traversal for furniture so each placement conditions on previously resolved structure and a reflective feedback loop to avoid error accumulation. After each object placement by VLM, we use the point cloud estimations to perform geometry-based refinement so that the rendered image aligns better with the input. HARMONY can produce 3D scenes that are semantically consistent and perceptually aligned with the reference image, extending single-image compositional reconstruction to complex indoor scene images. Experiments on synthetic and real-world images demonstrate that HARMONY outperforms the evaluated reconstruction baselines, while qualitative comparisons with GPT-6 Astra suggest more faithful object arrangements and better preservation of scene details.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.26793