Search: “3d-vision”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Recursive Code World Models: Building Complex Worlds through Recursive Scene Programs

RCWM reconstructs complex 3D worlds as executable code from a single image using recursive scene programs with global-local-global solver recursion.

The paper introduces Recursive Code World Models, coupling a Recursive Scene Program representation with a recursive construction solver for image-to-3D-world reconstruction. Each solver call establishes the whole scene, recursively reconstructs unresolved parts, and revisits the whole to refine composition, while a vision-language coding agent compares reference images with scene renders to guide refinement. RCWM outperforms prior code-based image-to-scene reconstruction methods, and ablations show deeper recursive calls improve fine-scale reconstruction.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research1

33

Learning 3D Editing without Paired Supervision via Generative Prior Distillation

New framework distills 2D editing and VLM priors into a feed-forward 3D editing model without paired 3D training data.

The method, PriorEdit3D, learns feed-forward instruction-guided 3D editing by distilling knowledge from foundation models instead of using ground-truth 3D pairs. Through a differentiable rendering pipeline it supervises a 2D visual prior from an image editing model at the main view and a Vision-Language Model semantic prior at novel views for instruction fidelity and identity preservation. A 3D-aware Distribution Matching regularization constrains outputs to the manifold of realistic 3D assets defined by a pretrained image-to-3D teacher. Experiments report superior instruction fidelity and cross-view consistency over state-of-the-art baselines, with code released on GitHub.

Hugging Face daily papers·Sep 3, 20:00 UTC · 13d agoAI research

30

Attention-DP3: Spatially Object-aware 3D Diffusion Policy via Geometry-aligned Attentional Conditioning

Attention-DP3 adds spatially object-aware attentional conditioning to 3D diffusion policies, improving robotic manipulation by up to 31% under heavy clutter.

Attention-DP3 injects object-level geometric cues into the unchanged DP3 diffusion policy via Tri-field Attentional Conditioning, using targetness, intra-target saliency, and backgroundness fields. Open-vocabulary 2D segmentation masks are lifted to 3D with calibrated camera geometry to build object-centric priors. Experiments on Adroit, DexArt, MetaWorld, and a real-world SO101 platform show state-of-the-art results, outperforming DP3 by up to 31% under heavy distractor clutter; the code is publicly available on GitHub.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research

15

RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting

RelightFormer is a feed-forward generative transformer for photorealistic single- and multi-view object relighting, trained on a 90K-object dataset.

Researchers introduce RelightFormer, a feed-forward generative transformer adapted from a video foundation model that performs direct image relighting without explicit intrinsic property estimation. The architecture injects target environment maps via a latent illumination module with cross-attention and uses permutation-invariant positional encodings for unordered multi-view inputs. Training relies on the newly constructed Laval Objaverse Dataset (LOD) with 90K objects and 39K unique illuminations, and the model shows state-of-the-art quality with strong zero-shot generalization across single-view, multi-view, and novel-view relighting.

Hugging Face daily papers·Sep 6, 20:00 UTC · 10d agoAI research

38

GoDeep: Annotation-Free Open-Vocabulary 3D Scene Understanding via Language-Space Lifting

GoDeep achieves annotation-free open-vocabulary 3D segmentation by grounding structured image descriptions in language-only embeddings, outperforming CLIP-based lifting on out-of-vocabulary objects.

GoDeep uses a vision-language model purely as a translator, producing structured entity-level image descriptions that are grounded, projected, and aggregated in a general-purpose language-only embedding space, with no 3D training corpus or dedicated 3D encoder required. On ScanNet++ the pipeline is competitive with strong annotation-free baselines, and on a cultural-heritage benchmark a systematic vocabulary correction reverses initial CLIP-based rankings. Language-space embeddings separate genuinely out-of-vocabulary objects more sharply, localize them within scenes, and keep all predictions explainable as discrete text.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:31 UTC · 8d agoAI research

26

Illusion of Depth: Revealing Hidden Stereo Vision Vulnerabilities in Depth Estimation

Simple repeating patterns let attackers shift stereo-camera depth estimates by up to 20 meters, triggering emergency braking in autonomous driving frameworks at 40 km/h.

The paper reveals an intrinsic vulnerability in stereo cameras stemming from pixel sampling and calibration processes, letting attackers finely control estimated depth of real obstacles using simple repeating patterns without adversarial ML techniques. The attack was evaluated against BM and SGBM stereo matching algorithms, deep learning models PSMNet, MoCha-Stereo, and UniMatch, the stereo-LiDAR fusion model SGM-DDC, and commercial cameras ZED2 and Intel RealSense D435; on ZED2, obstacles can be displaced up to 20 meters farther or 12 meters closer. A 0.5-second attack triggered emergency braking in a popular autonomous driving framework, with feasibility confirmed at driving speeds up to 40 km/h using CARLA. State-of-the-art defenses proved ineffective, and the authors propose a similarity-score strategy to dynamically detect and suppress depth discrepancies.

arXiv cs.CR·Sep 14, 20:51 UTC · 2d agoResearch

55

Hierarchical NeRF with JAX3D for Volumetric Rendering, Novel-View Synthesis, and 3D Reconstruction

MarkTechPost tutorial implements a hierarchical NeRF in JAX using jax3d volume-rendering primitives for novel-view synthesis and 3D reconstruction.

The tutorial builds an end-to-end hierarchical Neural Radiance Field using JAX, Flax, Optax, and jax3d's volume-rendering functions (sample_along_rays, volume_rendering, sample_piecewise_constant_pdf). It implements positional encoding, skip connections, separate coarse and fine networks, and view-direction conditioning with hierarchical importance sampling. Training uses JAX JIT compilation, Adam optimization, exponential learning-rate decay, and gradient clipping. Evaluation covers PSNR, depth and opacity visualization, 360-degree rendering, and marching-cubes geometry extraction.

MarkTechPost·Sep 13, 19:46 UTC · 3d agoAI research

18

3D Point Splatting for mmWave Radar Novel View Synthesis

Researchers propose 3DPS, a differentiable point renderer for mmWave radar novel view synthesis that outperforms optical-NVS baselines by 1.7x-5.2x.

The paper introduces 3D Point Splatting (3DPS), the first differentiable point renderer for radar, derived from the solid-angle form of the radar equation with ITU-R P.2040 material models and complex phasor splatting. On six outdoor ColoRadar scenes it reaches 0.587 mean Pearson correlation on held-out range-azimuth images, between 1.7x and 5.2x the RadarSplat, Radar Fields, and DART baselines. The same optimized scene produces ADC, complex range profile, and RA outputs via standard FFT pipelines, and training takes about 3 minutes per scene on an RTX 4090.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:52 UTC · 6d agoAI research

28

AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video

AlayaVista is a camera-controllable streaming video world model that decouples panoramic scene evolution from perspective synthesis, trained on a 1,318-hour 4K dataset.

AlayaVista builds a 360-degree scene prior from a single perspective image, evolves it as a camera-conditioned panoramic latent state, and maps it to perspective video via a latent viewport renderer plus a perspective refiner. Chunk-autoregressive generation and few-step distillation enable efficient streaming. The authors introduce MUGEN, a real-world panoramic video dataset with 1,318 hours of at-least-4K video and rich semantic and geometric annotations.

Hugging Face daily papers·Sep 12, 20:00 UTC · 4d agoAI research

45

CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs

CoVeR, a training-free coverage-based token pruner, preserves 93.5% of VLM 3D-reasoning performance using only about 8% of visual tokens.

Researchers introduce CoVeR, a deterministic, training-free selector that chooses visual tokens to cover every region of a multi-view 3D scene using only token coordinates. Unlike learned-importance and voxelization pruners, it enforces an exact per-scene token budget, avoids saturation plateaus, and prevents near-duplicate selections. Experiments across four vision-language models show it surpasses prior state of the art by 3.9 percentage points on average across three 3D reasoning benchmarks.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

45

SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem

Researchers release SpatialBlock-15k, a synthetic block-stacking dataset that improves 3D spatial reasoning in large vision-language models without dense geometric annotations.

The paper addresses limited spatial intelligence in LVLMs by training on structured block-manipulation tasks instead of costly real-scene annotated datasets. SpatialBlock-15k contains 15,000 synthetic problems covering 3D-to-2D projection, viewpoint transformation, and structural combination, with color modulation as visual cues. LVLMs trained on it via direct answering or reasoning-based prediction outperform baselines and generalize to real-world spatial tasks. Code and data are released on GitHub.

Hugging Face daily papers·Sep 6, 20:00 UTC · 10d agoAI research

28

Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation

Marigold V2 adapts diffusion transformers for monocular depth estimation, improving AbsRel 16-26% over the previous best on KITTI and ETH3D.

Huawei's Bayer lab revisits the Marigold approach to repurpose image generation and editing models built on the diffusion transformer (DiT) architecture into monocular depth estimators. The recipes target single-step inference from pretrained multi-step flow-matching models, with remedies including alignment to ground-truth semantic features and a two-stage fine-tuning protocol using a Sinkhorn-based loss. The resulting model produces crisper depth maps that generalize out-of-distribution and also achieves state-of-the-art results on surface normals estimation and intrinsic image decomposition.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research

42

Reflection-aware Generative Novel View Synthesis

Ref-GeNVS is a training-free method for reflection-consistent generative novel view synthesis that treats mirror images as two complementary views.

An arXiv paper proposes Ref-GeNVS, a training-free approach to generative novel view synthesis in scenes containing mirrors. It estimates the mirror plane, reflects camera poses to create virtual views, and applies mirror-gated attention plus reflection injection within a multi-view diffusion model. On synthetic and real mirror scenes, Ref-GeNVS outperforms recent generative NVS methods while requiring no fine-tuning.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:33 UTC · 12d agoAI research

26

VoT: Vision-of-Thought for Unified Multimodal Representation Alignment

Researchers propose Vision-of-Thought (VoT), a discrete visual-planning token layer between VLMs and diffusion transformers improving text-to-image semantic alignment.

VoT introduces a discrete visual-thinking layer between vision-language models and diffusion transformers, letting the VLM act as a multimodal planner that emits tokens describing objects and layouts before pixel generation. A specialized VoT tokenizer is trained with VLM alignment, feature reconstruction, and vector-quantization losses. Experiments show improved semantic alignment and a structured, interpretable interface for controllable generation.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:53 UTC · 9d agoAI research

24

TFTrack: A Template-Free Framework for Efficient 3D Point Cloud Tracking

Researchers propose TFTrack, a template-free LiDAR 3D single object tracking framework cutting FLOPs ~50% while running at ~120 FPS.

TFTrack is the first template-free framework for 3D Single Object Tracking, dropping template-search pairings and complex motion modeling in favor of the prior bounding box center plus geometric alignment. It ships in three variants (TFTrack-Voxel, TFTrack-Pillar, TFTrack-Point) covering sparse and dense 3D representations. On KITTI and nuScenes it is competitive with leading template-based trackers while reducing FLOPs by about 50% and running near 120 FPS. Code is released, targeting real-time deployment in embedded robotics such as autonomous vehicles.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 16:43 UTC · 9d agoAI research

20

Qwen-Drive 1.0 tells you why it brakes, just don't expect the explanation to match the maneuver

Alibaba's Qwen-Drive 1.0 adds 3D perception and planning modules to Qwen3.5-4B for driving tasks, though explanations often mismatch maneuvers.

Qwen-Drive 1.0, built on Qwen3.5-4B, combines spatial perception, traffic question answering, and route planning in one vision-language model, adding a bird's-eye-view perception module and a Planning Expert trained via staged fine-tuning and reinforcement learning. The paper finds text-image models do not inherently grasp 3D space; spatial accuracy only improved when the base vision-language model itself was trained on spatial tasks, while avoiding catastrophic forgetting of general knowledge. The cut reinforcement learning-trained version halved road-departure rate in simulation from 24% to 12%, and the model beats specialized driving models in most of Qwen's benchmarks, but its explanations sometimes conflate causes like distant red lights and crossing children, and results partly rest on self-designed tests. The work follows prior findings from PaLM-E and a UC Santa Cruz adversarial sign attack on DriveLM showing VLM driving models' reasoning and spatial gaps.

The Decoder·Sep 7, 12:15 UTC · 9d agoAI research

40

TransNormal-2: Geometry-Grounded Rectified Flow with Edge-Aware Decoding for Precise Normal Estimation

TransNormal-2 improves monocular surface-normal estimation by fixing VAE edge degradation with geometry-aware losses and refinement, matching MoGe-2 with 1.4% of annotations.

TransNormal-2 is a FLUX.2-based rectified-flow framework for monocular surface-normal estimation with single-step deterministic inference. The authors quantify that VAE 8x spatial compression introduces 1.3-8.5 degrees of mean angular error even on ground-truth normals, with edge error up to 2.8x the global error. The method adds geometry-aware pixel-space losses and an RGB-guided Geometric Refinement Module to correct boundary-localized decoding errors. It matches or exceeds MoGe-2 on all eight reported metrics using only 1.4% as many task-specific annotations, and cuts transparent-object MAE by 4.2 degrees on ClearGrasp and 3.1 degrees on ClearPose.

Hugging Face daily papers·Sep 5, 20:00 UTC · 11d agoAI research

20

Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy

Researchers added Greek to the Cosmos3 vision-language-action policy using only machine-rephrased instructions, finding bilingual training reaches roughly two fifths of English performance.

The paper studies localizing the open Cosmos3 vision-language-action robot policy to Greek without architectural changes, using machine-rephrased instructions only. Bilingual training yields a consistent 6.7-7.1 point margin over controls on a 90-task, three-seed evaluation suite, while Greek-only training gains at most 2.7 points. Several common evaluation instruments, including color-histogram metrics and single-goal benchmarks, produced false conclusions, and results were dominated by seed variation. The authors recommend building guaranteed-null baselines and replicating low-resource-language results across seeds.

Hugging Face daily papers·Sep 6, 20:00 UTC · 10d agoAI research

24

SenseNova-U1.5: Towards Native Unified Visual Intelligence

SenseTime releases SenseNova-U1.5, an 8B-MoT encoder-free multimodal model unifying visual understanding, reasoning, and generation with native 4K resolution.

SenseNova-U1.5 is an 8B mixture-of-transformers multimodal model with an encoder-free, VAE-free architecture that understands, reasons about, and generates visual content at native resolutions up to 4K. Post-training optimizes specialized experts for visual aesthetics, bilingual text rendering, infographic generation, and image editing, consolidated through multi-expert on-policy distillation. Evaluations report gains in image fidelity, text rendering, multi-reference editing, and instruction following. The team plans to open-source training code including supervised fine-tuning, reinforcement learning, and on-policy distillation.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoModel release

48

Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning

FactoSR factorizes 4D spatial reasoning into XY, Z, and T reinforcement-learning sub-objectives, boosting VLM performance on VSI-Bench by 5.9% and All-Angles-Bench by 4.5%.

Researchers present FactoSR, a factorized reinforcement learning framework that decomposes world-consistent reasoning into planar correspondence, depth consistency, and temporal reversibility sub-objectives. Optimizing these verifiable constraints turns the ill-posed projection recovery problem into tangible reasoning steps. Evaluations show gains of 5.9% on VSI-Bench and 4.5% on All-Angles-Bench for 3D and 4D reasoning, arguing VLMs' spatial bottleneck stems from training on 2D projections versus latent 3D geometry and temporal continuity.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research

26

Reason Through the Latent! Making Latent Visual Reasoning Necessary

Researchers introduce CVRR, forcing multimodal models to rely on recurrent latent computation rather than accessible image tokens, validated via causal interventions and benchmarks.

The paper presents Causal Visual Recurrent Reasoning (CVRR), which makes recurrent hidden-state computation the required image-conditioned path for prediction in vision-language models. Before decoding, visual states and the original multimodal KV cache are removed so only the final recurrent state carries image information to the answer. CVRR retains strong performance on V*, MMVP, BLINK, and MME-RealWorld-Lite while comparable latent reasoners fail under the same constraint. Causal interventions show predictions remain sensitive to recurrent content and that persistent visual evidence causally revises the recurrent trajectory.

Hugging Face daily papers·Sep 5, 20:00 UTC · 11d agoAI research

25

Can Edge-Deployable Vision-Language Models Identify Species?

Evaluation of 2-8B VLMs (Qwen3-VL, Gemma3) against BioCLIP on camera-trap species ID shows all models degrade sharply on field imagery.

The study tests whether edge-deployable 2-8B vision-language models carry genuine taxonomic knowledge, comparing Qwen3-VL 2B/4B/8B and Gemma3 4B against the 300M specialist BioCLIP on a 96-species task across clean iNaturalist photos and six LILA.science camera-trap collections. All models degrade 9.6-26.6 percentage points on field imagery, and BioCLIP outperforms every VLM by 33.2-59.2 points on an expanded 200-image sample, suggesting specialized data rather than scale drives the gap. Under open-set prompting, 5.9-9.6% of responses are syntactically valid but taxonomically nonexistent species names, with fabrication rankings replicating across evaluation sets.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:57 UTC · 6d agoAI research1

30

UniH^3: Unifying Hierarchical Homogeneity and Heterogeneity for All-in-One Medical Image Restoration

UniH3 unifies hierarchical homogeneity and heterogeneity modeling for all-in-one medical image restoration across modalities and degradation types.

UniH3 introduces a Hierarchical Homogeneity Memory module that distills shared anatomical priors from high-quality images, injected via a Homogeneity-Guided Attention mechanism. A Hierarchical Heterogeneity Balancer mitigates inter- and intra-task conflicts during multi-task optimization. It achieves state-of-the-art on MedIR-2D-500K and MedIR-3D-3D benchmarks for both all-in-one and single-task restoration, with code released on GitHub.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research

25

Feature Recovery for Object Understanding After Irreversible Fire Damage

TRACE benchmark with 21.4K scenes studies post-fire object understanding; a Feature Recovery Module improves degraded-image retrieval by 12.5% and material recovery by 20.1%.

The paper introduces TRACE, a transformation-aware benchmark with 21.4K real-image-grounded synthetic scenes, 499 object identities across 189 categories, and five tasks covering degraded-object detection, pristine-state recovery, material recovery, description generation, and functional reasoning. Existing models degrade sharply: RF-DETR mAP falls 71% relative from least to most severe level, and InternVL3.5 retrieval R@1 drops from 93.85 to 28.11. The proposed Feature Recovery Module maps degraded encoder features to pristine-aligned representations while keeping the host model frozen, averaging relative gains of 12.5% for retrieval and 20.1% for material recovery across VLM hosts and severity levels.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research

30

FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation

FLAT jointly trains a multimodal encoder with text-to-image and image-to-text decoders, producing flexible-length tokens that hit 83.1 GenEval on T2I after fine-tuning.

FLAT (Flexible-Length Aligned Transmodal representations) is a pre-training framework that jointly optimizes a shared multimodal encoder with T2I and I2T decoders, combining contrastive alignment with bidirectional cross-modal generative objectives. It maps visual and textual inputs into a unified continuous 1D sequence space and uses nested dropout over prefix-K tokens for dynamic output lengths. A single pre-training stage supports cross-modal retrieval and generation (71.1 GenEval), with task-specific fine-tuning reaching 83.1 GenEval on T2I, 40.5 BLEU-4 and 138.6 CIDEr on MS-COCO captioning, and strong Recall@5 on MS-COCO and Flickr30K.

Hugging Face daily papers·Sep 14, 20:00 UTC · 2d agoAI research1

35

SNAP3D: Physically Grounded 3D Parts for Assembly from a Single Image

SNAP3D uses physics simulation feedback to make single-image 3D part generation produce valid, stable assemblies, validated through 3D printing.

The framework improves part-aware 3D generation by resolving inter-part penetration, recovering contact graphs between neighboring parts, and placing parameterized connectors at contact surfaces. Physical simulation feedback refines connector placement, orientation, and dimensions to improve assembly stability while preserving geometry. A physics-based evaluation protocol tests assembly validity and stability under gravity, and results are validated through 3D printing and real-world assembly.

Hugging Face daily papers·Sep 10, 20:00 UTC · 6d agoAI research

15

TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model

Researchers introduce TANGO, a whole-body vision-language-action model enabling zero-shot language-guided humanoid navigation on the Unitree G1 robot.

TANGO addresses humanoid navigation in cluttered indoor environments by predicting 29-DoF joint-space actions directly from natural-language instructions and egocentric RGB, rather than 2D path planning. It is trained entirely in simulation via a pipeline combining global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. In simulation it achieves state-of-the-art vision-language navigation performance and transfers zero-shot to a Unitree G1 humanoid without any real-world navigation data.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research1

27

DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination

DeCAL, a contact-aware dexterous vision-language-action model with visuo-tactile fusion, reports 71% average task success.

DeCAL is a physically-grounded dexterous vision-language-action (VLA) model built on a Mixture-of-Transformers architecture with specialized experts for understanding, imagination, and action generation. It introduces Adaptive Visuo-Tactile Fusion with contact-aware gating and Visuo-Tactile Latent Co-Imagination to jointly model visual and tactile dynamics. It reports state-of-the-art results with a 71% average success rate and 83.4% progress success rate, plus generalization to unseen scenarios.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:47 UTC · 8d agoAI research1

30

Canonical Color as a Lens into Concept Decodability in Vision Encoders and VLMs

Probing study shows vision encoders make canonical color linearly decodable from grayscale images and tie it to object identity.

Researchers use canonical color as a controlled testbed for measuring conceptual (not just visible) information in vision encoder representations. A dataset of objects with canonical colors was built, and probes on both color and grayscale images show canonical color remains decodable even when color is removed from the input, linked to predicted object identity. Extending to full VLMs, they find post-training has a surprisingly large effect on color decodability in the vision encoder.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:50 UTC · 8d agoAI research1

27

SceneMosaic: Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolution

SceneMosaic combines image-based 3D priors with VLM agent refinement to generate diverse, simulation-ready indoor scenes 24x faster than agentic baselines.

SceneMosaic is a hybrid framework that takes an initial candidate from a learned image-to-3D prior and evolves it with VLM agents for efficiency and physical validity. It decomposes scenes into independent local units, evolves each separately, and composes the global scene via Cartesian product. On SceneEval-100 it matches the strongest agentic baseline in semantic layout quality with a 24x speedup, substantially reduces physical violations, and receives the highest human ratings. Code is publicly available.

Hugging Face daily papers·Sep 3, 20:00 UTC · 13d agoAI research

25