Search: “MovieGrid”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Multi-Grid Post-Training for Long-Form Multi-Shot Video Generation

MovieGrid arranges long videos on spatial grids during post-training, generating 6.05x more shots than temporal packing with state-of-the-art cross-shot consistency.

MovieGrid is a multi-grid post-training paradigm that decomposes long videos into temporally ordered chunks arranged on a spatial grid for joint modeling, enabling cross-chunk information exchange. The authors build the Multi-Grid Long Video (MGLV) dataset from 1,000 long-form videos, producing 54K grid videos paired with character-aware story prompts. Under the same token budget, MovieGrid generates 6.05x more shots than Temporal Packing in a 1,616-frame video. It achieves state-of-the-art intra-shot consistency of 0.9131 versus 0.8086 for HoloCine and inter-shot consistency of 0.5914 versus 0.5384 for StoryMem.

Hugging Face daily papers·Sep 5, 20:00 UTC · 10d agoAI research

20

PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control

PhysStream enables mid-generation interactive control of physics-grounded video via structured scene memory and velocity-increment signals, reducing motion distribution distance 33%.

PhysStream is an autoregressive physics-grounded image-to-video model that maintains structured scene memory—positional maps and object tracking maps derived online from previously generated frames—and accepts fine-grained motion control via sparse velocity-increment signals encoding physical quantities. Training runs in two stages: a bidirectional model finetuned with motion-control conditioning, then a causal autoregressive model with structured scene memory. It supports interactive mid-generation control over multi-object tabletop rigid-body scenes, reducing motion distribution distance (FVMD) by 33% and trajectory error by 12% over the strongest baselines. Human evaluators preferred it in over 85% of in-the-wild comparisons.

Hugging Face daily papers·updated Sep 15, 17:55 UTC · 1d ago·first Sep 14, 20:00 UTC · 1d agoAI research 2 sources

28

SceneMosaic: Efficient and Diverse Simulation-Ready Scene Generation via Hybrid Agentic Layout Evolution

SceneMosaic combines image-based 3D priors with VLM agent refinement to generate diverse, simulation-ready indoor scenes 24x faster than agentic baselines.

SceneMosaic is a hybrid framework that takes an initial candidate from a learned image-to-3D prior and evolves it with VLM agents for efficiency and physical validity. It decomposes scenes into independent local units, evolves each separately, and composes the global scene via Cartesian product. On SceneEval-100 it matches the strongest agentic baseline in semantic layout quality with a 24x speedup, substantially reduces physical violations, and receives the highest human ratings. Code is publicly available.

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research

25

RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting

RelightFormer is a feed-forward generative transformer for photorealistic single- and multi-view object relighting, trained on a 90K-object dataset.

Researchers introduce RelightFormer, a feed-forward generative transformer adapted from a video foundation model that performs direct image relighting without explicit intrinsic property estimation. The architecture injects target environment maps via a latent illumination module with cross-attention and uses permutation-invariant positional encodings for unordered multi-view inputs. Training relies on the newly constructed Laval Objaverse Dataset (LOD) with 90K objects and 39K unique illuminations, and the model shows state-of-the-art quality with strong zero-shot generalization across single-view, multi-view, and novel-view relighting.

Hugging Face daily papers·Sep 6, 20:00 UTC · 9d agoAI research

38

TempCloze: Can Video-LLMs Identify the Missing Middle?

TempCloze benchmark tests Video-LLMs' temporal reasoning with 1,521 videos, finding temporal alignment is the primary failure mode across 31 models.

TempCloze is a video cloze benchmark in which models must identify the true missing middle clip given the beginning and ending clips, using 1,521 carefully filtered videos from seven sources, mostly long-take and egocentric footage. Distractors are constructed along three dimensions: Semantic, Alignment and Progression, with shared scenes and objects to reduce appearance cues. Evaluation of 10 proprietary and 21 open-source Video-LLMs found Alignment is the primary bottleneck, with models often recognizing plausible semantics and local event progression but struggling with temporal alignment.

Hugging Face daily papers·Aug 31, 20:00 UTC · 15d agoAI research

25

AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video

AlayaVista is a camera-controllable streaming video world model that decouples panoramic scene evolution from perspective synthesis, trained on a 1,318-hour 4K dataset.

AlayaVista builds a 360-degree scene prior from a single perspective image, evolves it as a camera-conditioned panoramic latent state, and maps it to perspective video via a latent viewport renderer plus a perspective refiner. Chunk-autoregressive generation and few-step distillation enable efficient streaming. The authors introduce MUGEN, a real-world panoramic video dataset with 1,318 hours of at-least-4K video and rich semantic and geometric annotations.

Hugging Face daily papers·Sep 12, 20:00 UTC · 3d agoAI research

45

DriveZero: End-to-End Driving Beyond Human Demonstrations

DriveZero pairs a frozen vision-foundation-model perception stack with a PPO-trained closed-loop RL teacher to beat replay experts on nuPlan.

DriveZero is an end-to-end camera-only autonomous-driving planner that separates perception and action. Its DriveVFM perception backbone consolidates frozen vision foundation models (DINOv3, SigLIP2, SAM, Depth Anything V2) from raw images without task annotations, while DriveRL trains a privileged PPO teacher policy through closed-loop rollouts in interactive worlds built from real driving logs. The planner distills this teacher, achieving a 93.57 mean nuPlan score across Val14, Test14-hard and Test14-random splits and beating the Log-Replay expert on all three. It also sets state of the art on NAVSIMv1, NAVSIMv2 and closed-loop HUGSIM without human trajectory supervision.

Hugging Face daily papers·Sep 4, 20:00 UTC · 11d agoAI research

30

LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI Agents

LLaDA-UI, a 16.7B block-wise diffusion vision-language GUI agent, outperforms Qwen2.5-VL-7B and beats Qwen3-VL-8B on four of six GUI benchmarks.

LLaDA-UI is a 16.7B-parameter MoE-based, block-wise diffusion vision-language GUI agent built on the LLaDA2.0-mini-base diffusion language backbone with a native-resolution vision encoder. It uses a two-stage pipeline: general multimodal pre-training followed by GUI-agent supervised fine-tuning on mobile, desktop, web, and grounding data. It substantially outperforms Qwen2.5-VL-7B and surpasses Qwen3-VL-8B on four of six reported GUI benchmarks, establishing block-wise diffusion as a practical paradigm for latency-sensitive multimodal agents.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

48

RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives

RenderFormer-V2 is a transformer-based neural renderer handling caustics, volumetric scattering and out-of-distribution materials without per-scene training or specialized code.

RenderFormer-V2 is a learned transformer-based neural rendering model that models global light transport as a sequence-to-sequence transformation, handling caustics, volumetric scattering, environment lighting, textured and displaced surfaces and out-of-distribution materials. It uses a two-stage process: a view-independent stage resolving primitive-to-primitive transport, and a view-dependent stage converting the neural scene representation into pixels. Improvements include combined windowed-attention with a rendering-informed attention sink for scalability, support for heterogeneous primitives like environment maps and participating media, and a surface-reflectance-independent neural material encoding, validated across diverse scenes with extensive ablations.

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research

22

Label-Guided Knowledge Distillation for 3D-CNNs in Action Recognition

LGKD uses ground-truth labels to guide feature distillation for 3D-CNNs, combining sample-wise and class-wise distillation for action recognition.

The paper proposes Label-Guided Knowledge Distillation (LGKD) for 3D-CNNs, noting that most video feature distillation methods are simple adaptations of image techniques that neglect temporal-dimension differences. LGKD combines sample-wise distillation, which uses label information and the teacher's probability distribution to guide features impacting temporal accuracy, with class-wise distillation employing a prototype network to capture relational knowledge among same-category samples. Experiments on the UCF101 and HMDB51 action recognition benchmarks achieve competitive results.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 16:21 UTC · 5d agoAI research

15

Show-Harness: Just a VLM Agent Can Play Robots

Show-Harness lets VLM agents control robots via discrete semantic action units, outperforming VLA baselines zero-shot and after light fine-tuning.

Show-Harness is an embodied agent harness that exposes discrete semantic action units a VLM reasons over, with embodiment-specific interpreters grounding them into local robot actions. It enables zero-shot robot control with closed-source frontier VLMs and low-cost adaptation of small open-source VLMs using only a few GPU-hours of fine-tuning. The companion GUMI (GUI Manipulation Interface) extends the same semantic action space to GUI-based demonstration collection without specialized teleoperation hardware. Experiments show robust generalization across tasks, embodiments, and environments, beating representative agentic and VLA paradigms.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 17:53 UTC · 7d agoAI research

32

ReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation

ReMoMask-2 rebuilds retrieval in the generator's latent space for text-to-motion generation, achieving lowest FID on KIT-ML and SnapMoGen.

ReMoMask-2 is a retrieval-augmented text-to-motion framework that constructs its retrieval database directly in the generator's pre-quantization latent space and aligns text queries through a distilled lightweight projector, eliminating the representation gap. The framework combines Hierarchical Bidirectional Momentum contrastive learning, Semantic Spatial-Temporal Attention, and Topology Structured Masking to handle hierarchical motion structure. The retriever achieves state-of-the-art accuracy, and ReMoMask-2 attains the lowest FID on KIT-ML and SnapMoGen, with a single mask-transformer stage outperforming the previous two-stage pipeline while delivering the fastest inference.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

25

CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements

Researchers release CosmoH2G, a 6,189-episode hand-to-gripper dataset with a two-stage method for complex spatial robot manipulation.

The paper introduces a scalable acquisition pipeline using a handheld gripper to collect paired hand-gripper demonstrations, producing 6,189 episodes across 1,254 unique objects with higher spatial complexity than existing benchmarks. A two-stage framework first predicts sparse gripper keyframes (initial and terminal), then generates the full continuous action sequence conditioned on them, while learning gripper orientation and post-optimizing translation via grasping heuristics and kinematic consistency. Simulation and real-robot experiments show stable, precise hand-to-gripper transfer of complex spatial manipulations, outperforming traditional baselines.

Hugging Face daily papers·Sep 6, 20:00 UTC · 9d agoAI research

24

CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs

CoVeR, a training-free coverage-based token pruner, preserves 93.5% of VLM 3D-reasoning performance using only about 8% of visual tokens.

Researchers introduce CoVeR, a deterministic, training-free selector that chooses visual tokens to cover every region of a multi-view 3D scene using only token coordinates. Unlike learned-importance and voxelization pruners, it enforces an exact per-scene token budget, avoids saturation plateaus, and prevents near-duplicate selections. Experiments across four vision-language models show it surpasses prior state of the art by 3.9 percentage points on average across three 3D reasoning benchmarks.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

45

TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents

TRACE, a training-free visual token pruning framework, cuts GUI agent inference latency and memory while keeping trajectory-wide visual evidence reusable.

TRACE is a training-free framework for trajectory-robust admission and coverage-aware evidence ordering that prunes high-resolution screenshot tokens accumulated in GUI agent trajectories. It ranks visual evidence using a query-independent layout-derived interaction prior combined with instruction relevance and feature novelty, and reserves part of the budget for native tokens distributed across the screen to repair spatial coverage. A monotone KV contraction incrementally compresses retired frames into compact session state, avoiding repeated visual encoding or pruning. Experiments across six GUI benchmarks and diverse models verify effectiveness under tight budgets, with source code to be released.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

28

One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing

Researchers introduce EditVid, a training-free video editing framework scoring 78.16 FiVE-Acc versus 58.95 for the strongest comparable baseline.

EditVid is a unified training-free framework for diverse instruction-guided and subject-guided video editing. It combines sparse causal memory for local coherence, correspondence-based post-attention token injection for long-range identity preservation, and soft latent blending for edit locality. On the FiVE benchmark it reaches 78.16 FiVE-Acc against 58.95 for the strongest evaluated training-free baseline, with competitive results on IVEBench. A user study showed 51.8% overall preference for EditVid over 7 competing methods.

Hugging Face daily papers·Sep 2, 20:00 UTC · 13d agoAI research

38

Agentic Visual Generation: From Generative Models to Agentic Control

Researchers propose an L0-L4 control taxonomy for agentic visual generation, classifying controllers from fixed conditioning to experience-adaptive decision-making.

This paper proposes a taxonomy for agentic visual generation organized by what the controller can directly control in the generation process, rather than by planning depth, tool count, or model size. Levels range from L1 Conditioning Control through L2 Execution Control, L3 Outcome-Adaptive Control, and L4 Experience-Adaptive Control, with L0 Fixed Support denoting systems without a deployed decision-making controller. The framework is applied across image, video, editing, 3D, world, slide, and user-interface generation to map how controller capabilities and mechanisms have evolved across the field.

Hugging Face daily papers·Sep 5, 20:00 UTC · 10d agoAI research

22

Hierarchical NeRF with JAX3D for Volumetric Rendering, Novel-View Synthesis, and 3D Reconstruction

MarkTechPost tutorial implements a hierarchical NeRF in JAX using jax3d volume-rendering primitives for novel-view synthesis and 3D reconstruction.

The tutorial builds an end-to-end hierarchical Neural Radiance Field using JAX, Flax, Optax, and jax3d's volume-rendering functions (sample_along_rays, volume_rendering, sample_piecewise_constant_pdf). It implements positional encoding, skip connections, separate coarse and fine networks, and view-direction conditioning with hierarchical importance sampling. Training uses JAX JIT compilation, Adam optimization, exponential learning-rate decay, and gradient clipping. Evaluation covers PSNR, depth and opacity visualization, 360-degree rendering, and marching-cubes geometry extraction.

MarkTechPost·Sep 13, 19:46 UTC · 2d agoAI research

18

OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolution

OracleZoom enables recursive extreme-scale image super-resolution via reference-constrained on-policy distillation, reducing hallucinations at deep zoom scales.

OracleZoom tackles recursive super-resolution, where repeated feeding of predictions back into the same model leaves deeper-scale outputs unsupervised as required source resolution grows geometrically. The framework trains on its own trajectory while carrying the last ground-truth evidence beyond the supervision boundary, combining direct and cross-scale supervision, a no-reference quality objective, a KL-constrained pretrained latent prior, and EMA consistency. Across seven datasets it achieves state-of-the-art SR quality across zoom scales, averaging 0.713 CLIPIQA with larger gains at deeper scales and significantly reduced hallucinations. Code, data, and models are publicly released.

Hugging Face daily papers·Sep 5, 20:00 UTC · 10d agoAI research

20

GoDeep: Annotation-Free Open-Vocabulary 3D Scene Understanding via Language-Space Lifting

GoDeep achieves annotation-free open-vocabulary 3D segmentation by grounding structured image descriptions in language-only embeddings, outperforming CLIP-based lifting on out-of-vocabulary objects.

GoDeep uses a vision-language model purely as a translator, producing structured entity-level image descriptions that are grounded, projected, and aggregated in a general-purpose language-only embedding space, with no 3D training corpus or dedicated 3D encoder required. On ScanNet++ the pipeline is competitive with strong annotation-free baselines, and on a cultural-heritage benchmark a systematic vocabulary correction reverses initial CLIP-based rankings. Language-space embeddings separate genuinely out-of-vocabulary objects more sharply, localize them within scenes, and keep all predictions explainable as discrete text.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:31 UTC · 8d agoAI research

26

BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender

Blender-VideoBench evaluates agentic video understanding by having agents programmatically reconstruct real videos in Blender scenes.

BVB (Blender-VideoBench) tests whether multimodal agents truly understand videos by requiring programmatic reconstruction of real-world videos as animated Blender scenes via a lightweight Mini-BVB harness under identical sandbox and cost constraints. Evaluation uses Dual VQA for spatiotemporal fact preservation and Latent Similarity for perceptual match, combined in a square-root mean overall score. Across 51 configurations from 10 model families, the best model reaches 88.6 Latent Similarity but retains only 53.7% of source-correct spatiotemporal answers, showing semantic retention remains the main challenge.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI research

30

ActionSplice: In-Flight Action Editing for Interactive World Models

ActionSplice enables in-flight action editing in chunk-autoregressive video world models via a lightweight corrector, avoiding rollback or waiting for the next chunk.

ActionSplice is an inference framework that formulates in-flight action editing for chunk-autoregressive video world models as Counterfactual State Transport (CST), where a lightweight corrector transports the interrupted backbone-native representation toward the matched state induced by the revised action. The world model and sampler remain frozen, and sampling resumes without replaying completed evaluations. Across minWM-Wan Action2V and HY-WM1.5, the retargeting variant CST-R reduces rollback-relative LPIPS by 61.5% and 75.9% versus direct condition swapping, while the temporal-splicing variant CST-T reduces suffix LPIPS by 56.1% and 77.5% with 2.73x and 1.69x pixel-ready speedups over waiting.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

22

What Matters, When? Diagnosing and Improving Conditional Visual Grounding in Visuomotor Imitation Policies

Researchers diagnose conditional visual grounding failures in visuomotor imitation policies and show targeted interventions substantially improve distractor robustness.

The paper studies why ACT-based visuomotor imitation policies fail when visually similar distractor objects or receptacles are introduced, finding sensitivity depends on both distractor type and manipulation stage. Interventions including distractor augmentation, phase-dependent attention regularization, and appearance-based visual prompting improve target selection while preserving spatial control information, with gains in simulation and on a physical UR3e. The same failure pattern is confirmed in a pretrained vision-language-action policy on a state-conditioned medical instrument-handling task.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:27 UTC · 12d agoAI research

18

Inside NVIDIA’s cuDNN Graph API: Fusion, Autotuning, and Plan Reuse with cuDNN Frontend

MarkTechPost tutorial walks through NVIDIA's cuDNN Frontend graph API, covering kernel fusion, autotuning, plan reuse, and CUDA graph capture on Colab GPUs.

The tutorial explains how to express GPU computations as operation graphs via the cuDNN Frontend graph API, running the five-step build pipeline of validate, build operation graph, create execution plans, check support, and build plans. It progresses from a single fused convolution with bias and ReLU to autotuning across engine configs, FP8-style epilogues, attention, plan serialization, dynamic shapes, and CUDA graph capture. Each kernel is benchmarked against a PyTorch reference on a single Colab GPU to verify correctness and measure cost. The piece also covers practical setup issues like making libcudnn.so visible to the frontend's dynamic loader.

MarkTechPost·Sep 15, 21:37 UTC · 21h agoAI tools & infra1

18

Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models

Latent Interface Training improves robot foundation model generalization by constraining visual conditioning, boosting LIBERO-Plus success up to 10.7 points.

The paper identifies vision-action shortcuts where robot policies exploit task-irrelevant visual cues that fail under distribution shift. Latent Interface Training (LIT) first trains an action expert conditioned on language, robot state, and terminal SE(3) end-effector poses without images, then constrains visual input through a pose-supervised latent interface. Across four VLA and world-action architectures (Pi0.5, MolmoAct2, FAST-WAM, ImageWAM), LIT improves LIBERO-Plus success by 3.87-10.70 percentage points. Real-world tests show 13.30-16.70 percentage-point gains under unseen cameras, lighting, and distractors.

Hugging Face daily papers·Sep 10, 20:00 UTC · 5d agoAI research

32

Recursive Code World Models: Building Complex Worlds through Recursive Scene Programs

RCWM reconstructs complex 3D worlds as executable code from a single image using recursive scene programs with global-local-global solver recursion.

The paper introduces Recursive Code World Models, coupling a Recursive Scene Program representation with a recursive construction solver for image-to-3D-world reconstruction. Each solver call establishes the whole scene, recursively reconstructs unresolved parts, and revisits the whole to refine composition, while a vision-language coding agent compares reference images with scene renders to guide refinement. RCWM outperforms prior code-based image-to-scene reconstruction methods, and ablations show deeper recursive calls improve fine-scale reconstruction.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research1

33

GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

GE-Act 2.0 is a from-scratch pretrained world-action model for robotic manipulation, with success rising from 17.1% to 44.1% as co-training data scales to 30,000 hours.

Genie Envisioner Act 2.0 (GE-Act 2.0) is a world-action model whose generative and action components are all initialized from scratch on manipulation data, combining a control-oriented autoencoder (CoAE), single-step visual planner (SVP), and inverse dynamics model (IDM) trained jointly via knowledge-aligned selective optimization (KASO). Scaling co-training data from 300 to 30,000 hours raises zero-shot success from 17.1% to 44.1% on G1-OP and 13.4% to 31.1% on G2-90D, despite the latter comprising under 2% of data, suggesting cross-embodiment transfer. Gains span 19/20 and 18/20 skill groups, and skill-specific coverage correlates with zero-shot OOD success (Pearson r=0.80).

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research

30

World in World: Explore the World with World Models

World in World is a training-free inference-time interface enabling camera-controlled rerendering, long-horizon revisiting, and motion transfer with frozen causal video world models.

The framework converts heterogeneous control evidence into camera- and time-labelled clean visual states that are read through the native self-attention of a frozen causal video model. Evidence includes source-video observations, target-view scene projections, geometry renderings for completing newly exposed regions, and retrieved generated states beyond the rolling cache. A correspondence router combines persistent point identities with geometry for token correspondences, while evidence-wise attention CFG independently regulates each auxiliary channel. The shared interface supports camera-controlled rerendering, long-horizon revisiting, and human-motion transfer without additional training.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

30

TFTrack: A Template-Free Framework for Efficient 3D Point Cloud Tracking

Researchers propose TFTrack, a template-free LiDAR 3D single object tracking framework cutting FLOPs ~50% while running at ~120 FPS.

TFTrack is the first template-free framework for 3D Single Object Tracking, dropping template-search pairings and complex motion modeling in favor of the prior bounding box center plus geometric alignment. It ships in three variants (TFTrack-Voxel, TFTrack-Pillar, TFTrack-Point) covering sparse and dense 3D representations. On KITTI and nuScenes it is competitive with leading template-based trackers while reducing FLOPs by about 50% and running near 120 FPS. Code is released, targeting real-time deployment in embedded robotics such as autonomous vehicles.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 16:43 UTC · 9d agoAI research

20