Search: “motion-generation”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control

PhysStream enables mid-generation interactive control of physics-grounded video via structured scene memory and velocity-increment signals, reducing motion distribution distance 33%.

PhysStream is an autoregressive physics-grounded image-to-video model that maintains structured scene memory—positional maps and object tracking maps derived online from previously generated frames—and accepts fine-grained motion control via sparse velocity-increment signals encoding physical quantities. Training runs in two stages: a bidirectional model finetuned with motion-control conditioning, then a causal autoregressive model with structured scene memory. It supports interactive mid-generation control over multi-object tabletop rigid-body scenes, reducing motion distribution distance (FVMD) by 33% and trajectory error by 12% over the strongest baselines. Human evaluators preferred it in over 85% of in-the-wild comparisons.

Hugging Face daily papers·updated Sep 15, 17:55 UTC · 18h ago·first Sep 14, 20:00 UTC · 1d agoAI research 2 sources

28

Agentic Visual Generation: From Generative Models to Agentic Control

Researchers propose an L0-L4 control taxonomy for agentic visual generation, classifying controllers from fixed conditioning to experience-adaptive decision-making.

This paper proposes a taxonomy for agentic visual generation organized by what the controller can directly control in the generation process, rather than by planning depth, tool count, or model size. Levels range from L1 Conditioning Control through L2 Execution Control, L3 Outcome-Adaptive Control, and L4 Experience-Adaptive Control, with L0 Fixed Support denoting systems without a deployed decision-making controller. The framework is applied across image, video, editing, 3D, world, slide, and user-interface generation to map how controller capabilities and mechanisms have evolved across the field.

Hugging Face daily papers·Sep 5, 20:00 UTC · 10d agoAI research

22

A Chosen Future Can Still Be Rewritten: Causal Writability in Video Models

Study shows video models often learn correct physics but fail to use it; low-dimensional 'causal writability' edits can restore correct motion.

The paper demonstrates 'causal writability' in video generation models: physically correct motion remains available inside the model even when the model outputs incorrect motion. In a red/blue mass oscillation setup, a low-dimensional edit predicted from simple physical variables restores correct fast motion, with a sharp depth boundary marking commitment. Early causal writability predicts which training errors later get corrected, and both writability and closure reproduce in a pretrained 1.3B video model.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:58 UTC · 1d agoAI research

33

Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

Researchers introduce Motion-Omni, an end-to-end model generating speech with synchronized full-body motion, responding 5.4x faster than cascade pipelines.

Motion-Omni is an end-to-end framework in which a spoken dialogue model outputs facial expressions and hand, upper-body, and lower-body motion directly from the hidden states that produce speech, replacing two-stage cascade pipelines. Trained on 422,856 quality-ranked pseudo-labeled pairs (1,402 hours) with a Qwen2.5-7B-Instruct backbone, Motion-Omni-Q7 matches its teacher cascade within 2% on reference-free motion metrics, achieves a 2.62% word error rate, and runs faster than real time (RTF=0.78). The authors also release the SwDA-500 dataset and the first public evaluation protocol for stochastic open-ended full-body spoken dialogue.

Hugging Face daily papers·Aug 27, 20:00 UTC · 19d agoAI research1

45

UniMate: One Unified Model to Animate Diverse Skeletons

Researchers introduce UniMate, a topology-aware diffusion transformer generating text-driven motion for arbitrary 3D skeletons without per-skeleton retraining.

UniMate is a unified foundation model that synthesizes articulated motion for arbitrary skeletons from a rigged 3D asset and a text prompt, with no test-time optimization or per-skeleton fine-tuning. It uses a topology-aware diffusion transformer combining graph-aware attention bias, a spectral rotary position embedding generalizing RoPE via the graph Laplacian, and a rest-pose topological conditioner. Trained on UniML3D, a curated set of 13,006 motion sequences spanning bipedal to serpentine skeletons, it outperforms state-of-the-art baselines and supports zero-shot cross-topology transfer, in-betweening, and text-guided editing.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:59 UTC · 11d agoAI research

16

CosmoH2G: A Hand-to-Gripper Transfer Dataset and Baseline Method for Object Manipulation with Complex Spatial Movements

Researchers release CosmoH2G, a 6,189-episode hand-to-gripper dataset with a two-stage method for complex spatial robot manipulation.

The paper introduces a scalable acquisition pipeline using a handheld gripper to collect paired hand-gripper demonstrations, producing 6,189 episodes across 1,254 unique objects with higher spatial complexity than existing benchmarks. A two-stage framework first predicts sparse gripper keyframes (initial and terminal), then generates the full continuous action sequence conditioned on them, while learning gripper orientation and post-optimizing translation via grasping heuristics and kinematic consistency. Simulation and real-robot experiments show stable, precise hand-to-gripper transfer of complex spatial manipulations, outperforming traditional baselines.

Hugging Face daily papers·Sep 6, 20:00 UTC · 9d agoAI research

24

ReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation

ReMoMask-2 rebuilds retrieval in the generator's latent space for text-to-motion generation, achieving lowest FID on KIT-ML and SnapMoGen.

ReMoMask-2 is a retrieval-augmented text-to-motion framework that constructs its retrieval database directly in the generator's pre-quantization latent space and aligns text queries through a distilled lightweight projector, eliminating the representation gap. The framework combines Hierarchical Bidirectional Momentum contrastive learning, Semantic Spatial-Temporal Attention, and Topology Structured Masking to handle hierarchical motion structure. The retriever achieves state-of-the-art accuracy, and ReMoMask-2 attains the lowest FID on KIT-ML and SnapMoGen, with a single mask-transformer stage outperforming the previous two-stage pipeline while delivering the fastest inference.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

25

Multi-Grid Post-Training for Long-Form Multi-Shot Video Generation

MovieGrid arranges long videos on spatial grids during post-training, generating 6.05x more shots than temporal packing with state-of-the-art cross-shot consistency.

MovieGrid is a multi-grid post-training paradigm that decomposes long videos into temporally ordered chunks arranged on a spatial grid for joint modeling, enabling cross-chunk information exchange. The authors build the Multi-Grid Long Video (MGLV) dataset from 1,000 long-form videos, producing 54K grid videos paired with character-aware story prompts. Under the same token budget, MovieGrid generates 6.05x more shots than Temporal Packing in a 1,616-frame video. It achieves state-of-the-art intra-shot consistency of 0.9131 versus 0.8086 for HoloCine and inter-shot consistency of 0.5914 versus 0.5384 for StoryMem.

Hugging Face daily papers·Sep 5, 20:00 UTC · 10d agoAI research

20

Puppeteer: Object-Grounded Posture-Aware Co-Speech Gesture Generation

Researchers present Puppeteer, a posture-aware, object-grounded diffusion model generating physically consistent co-speech gestures with temporal control.

Puppeteer decomposes long gestures into structured primitives encoded by a causal variational autoencoder into temporally ordered latent tokens. Conditional diffusion in the causal latent space conditions on speech signals, motion history, an initial posture reference, and object geometry to synthesize physically consistent gestures. The authors also introduce new evaluation metrics and release SceneGes, the first curated synthetic 3D dataset of embodied co-speech gestures with corresponding 3D objects.

Hugging Face daily papers·Aug 30, 20:00 UTC · 16d agoAI research1

18

PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models

PhysBrain 1.5, an 8B physical foundation model, sets open-source state of the art across 28 embodied understanding benchmarks.

The paper presents PhysBrain 1.5, a unified 8B model for understanding physical environments, generating actions, and predicting future states, built from a vision-language model with joint autoregressive next-token prediction over language, end-effector motion, and dense visual targets. Pre-training uses embodied supervision from human interaction videos, followed by supervised fine-tuning on human demonstrations, robot trajectories, and simulated experience. The model averages 72.5 across 28 embodied benchmarks, setting a new open-source state of the art and performing on par with proprietary GPT-6-Astra and Gemini 3.6 Flash, with best open-source results on 14 benchmarks.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI research1

45

Kaininja: Extending Native 3D Generators to the Part Level

KaiNinja extends TRELLIS.2 native 3D generation to part-level assets via a dual-volume O-Voxel representation, cutting whole-object Chamfer distance by 40%.

KaiNinja extends the TRELLIS.2 native 3D generator to produce part-level assets instead of one fused mesh, enabling downstream editing, rigging, and simulation. A dual-volume form of the O-Voxel representation solves the problem that a single volume cannot represent interfaces where two parts touch. The model needs no segmentation network, is partly trained on LLM-agent-authored part data, lowers whole-object Chamfer distance by 40%, and raises strict part F-score by 16% versus other part-generation pipelines.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI research

18

LynnReal-Omni: Native multi-modal Video Generation for Agentic Visual Workflows

LynnReal-Omni unifies controllable video generation tasks in a 32B multimodal diffusion transformer, with a 27B Flash variant rendering 540p clips in 377 ms.

LynnReal-Omni is a native multimodal video generation framework built on a 32B shared multimodal diffusion transformer unifying text-to-video, image-conditioned generation, reference guidance, structural control, editing, restoration and long-video generation, accepting heterogeneous inputs like 3D renders and game recordings for agentic visual workflows. A dedicated 27B Flash model enables real-time rendering, producing a 22-frame 540p video in 377 ms on one H100 versus 843 ms for the full model. The work introduces a curated multi-shot audiovisual data pipeline and MSAVP, a 100-prompt, 20-metric evaluation design covering instruction following, plausibility, visual quality, temporal behavior and audio coordination.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI research

28

UniMate: One Unified Model to Animate Diverse Skeletons

UniMate is a topology-aware diffusion transformer generating articulated motion for arbitrary rigged skeletons from text, trained on 13,006 motion sequences.

UniMate is a unified foundation model that animates arbitrary rigged 3D skeletons from an asset and text prompt with no test-time optimization or per-skeleton retraining. It uses a topology-aware diffusion transformer combining graph-aware attention bias from joint relations and geodesic distances, a spectral rotary position embedding generalizing RoPE to kinematic trees via the graph Laplacian, and a global topological conditioner. The accompanying UniML3D dataset spans 13,006 motion sequences across bipedal, quadrupedal, avian, marine, insectoid, serpentine, and articulated rigid-object skeletons; the model outperforms baselines and supports zero-shot cross-topology transfer, in-betweening, expansion, and text-guided editing.

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research

30

TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model

Researchers introduce TANGO, a whole-body vision-language-action model enabling zero-shot language-guided humanoid navigation on the Unitree G1 robot.

TANGO addresses humanoid navigation in cluttered indoor environments by predicting 29-DoF joint-space actions directly from natural-language instructions and egocentric RGB, rather than 2D path planning. It is trained entirely in simulation via a pipeline combining global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. In simulation it achieves state-of-the-art vision-language navigation performance and transfers zero-shot to a Unitree G1 humanoid without any real-world navigation data.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research1

27

AgenticGen: Reward-Guided Agentic Video Generation for Advertising

AgenticGen applies DPO and GRPO reward-guided reasoning to ad video generation, improving TikTok CTR 2.72%, CVR 2.63%, and Advv 9.61%.

AgenticGen decomposes advertising video generation into two trainable reasoning stages, strategy selection and draft generation, supervised by online business feedback. It learns a performance-based reward from accumulated online feedback plus a rubric-based reward aligned with human quality standards, then optimizes policies with DPO followed by GRPO using process and outcome rewards. Online A/B experiments in the TikTok advertising system show CTR up 2.72%, CVR up 2.63%, and Advv up 9.61% over an SFT baseline.

Hugging Face daily papers·Aug 30, 20:00 UTC · 16d agoAI research

26

Reflection-aware Generative Novel View Synthesis

Ref-GeNVS is a training-free method for reflection-consistent generative novel view synthesis that treats mirror images as two complementary views.

An arXiv paper proposes Ref-GeNVS, a training-free approach to generative novel view synthesis in scenes containing mirrors. It estimates the mirror plane, reflects camera poses to create virtual views, and applies mirror-gated attention plus reflection injection within a multi-view diffusion model. On synthetic and real mirror scenes, Ref-GeNVS outperforms recent generative NVS methods while requiring no fine-tuning.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:33 UTC · 11d agoAI research

26

TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model

Researchers present TANGO, a whole-body vision-language-action model enabling humanoid robots to traverse cluttered spaces from language instructions.

TANGO predicts 29-DoF joint-space actions from egocentric RGB observations and natural-language instructions for whole-body humanoid navigation, going beyond 2D path planning. It is trained entirely in simulation using global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. The model reports state-of-the-art simulation performance and was deployed zero-shot on a Unitree G1 humanoid without any real-world navigation training data.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:59 UTC · 7d agoAI research1

28

GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

GE-Act 2.0 is a from-scratch pretrained world-action model for robotic manipulation, with success rising from 17.1% to 44.1% as co-training data scales to 30,000 hours.

Genie Envisioner Act 2.0 (GE-Act 2.0) is a world-action model whose generative and action components are all initialized from scratch on manipulation data, combining a control-oriented autoencoder (CoAE), single-step visual planner (SVP), and inverse dynamics model (IDM) trained jointly via knowledge-aligned selective optimization (KASO). Scaling co-training data from 300 to 30,000 hours raises zero-shot success from 17.1% to 44.1% on G1-OP and 13.4% to 31.1% on G2-90D, despite the latter comprising under 2% of data, suggesting cross-embodiment transfer. Gains span 19/20 and 18/20 skill groups, and skill-specific coverage correlates with zero-shot OOD success (Pearson r=0.80).

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research

30

Vidu S2: Real-Time Interactive, Editable, and Spatial Video Generation

Vidu S2 delivers real-time 720p interactive avatar generation plus real-time video editing with style, clothing, character, and background replacement.

Vidu S2 comprises Vidu S2-Avatar, a real-time interactive digital-character model, and Vidu S2-Editing, a real-time video editing model. S2-Avatar supports real-time 720p generation, dynamic references updatable at any moment, and stronger instruction following such as dancing, while S2-Editing performs real-time style rendering, clothing replacement, character replacement, and background replacement. The work also explores real-time spatial video generation for both models, reports outperforming all baselines, and offers a playable online demo at vidu.com.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoModel release

40

Diffusion TV: Experiencing Diffusion Models through Tangible, Embodied Interaction

Researchers present Diffusion TV, a CRT-based installation where antenna manipulation lets audiences physically experience diffusion model denoising.

Diffusion TV is an interactive installation built around a modified CRT television where turning the antenna controls the clarity of AI-generated images and sounds, mirroring the denoising process of diffusion models. Three channels present AI-generated animals from the past, present, and future within a temporal and ecological narrative. The authors frame the work as an embodied, non-verbal alternative to explainable AI that highlights intermediate generative states rather than final outputs.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:51 UTC · 11d agoAI research

12

Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models

Empirical study finds direct whole-file generation beats iterative diff-based editing for Flutter/Dart code models on about 1,790 held-out tasks.

Researchers trained Rainbow-Pony-100M from scratch and fine-tuned Qwen2.5-Coder-0.5B in both direct-generation and diff-based regimes, then evaluated four resulting models on roughly 1,790 Flutter/Dart tasks. Direct generation outperformed diff-based generation on compilation pass rate, bits-per-byte, character-level similarity, and blinded LLM-judge ratings. Diff-based editing is competitive only on short, localized edits in refactoring and error-handling tasks, a property the authors call task locality.

Hugging Face daily papers·Sep 4, 20:00 UTC · 11d agoAI research1

30

SynthGait-19K: A Physically Grounded Synthetic Video Dataset for Gait Parameter Estimation

Researchers release SynthGait-19K, a synthetic video dataset with 19,272 walking videos for training gait parameter estimation models.

SynthGait-19K is a physically grounded synthetic video dataset built from 6,427 MoCap sequences of 437 subjects, yielding 19,272 walking videos with SMPL motion and annotations for six gait parameters. The authors introduce Gait2Vid, a pipeline that unifies heterogeneous MoCap recordings and synthesizes RGB videos under controllable viewpoints, validating gait events against force-platform measurements. Using the dataset they benchmark direct RGB, pose-based, biomechanical, and human-mesh-recovery approaches, and introduce GaitXFormer as a direct RGB reference model. Findings show synthetic supervision transfers to real video, while spatial gait parameters are more sensitive to visual domain shift.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

28

StepAudio 3 Gen Technical Report

StepAudio 3 Gen unifies TTS, voice design, music, and sound effects via discrete autoregressive modeling over RVQ tokens.

StepAudio 3 Gen is a general-purpose audio generation model covering zero-shot TTS, voice design, vocal generation, sound effects, music, vibe speech, and mixed audio in one framework. It uses discrete autoregressive modeling over residual vector quantization (RVQ) tokens rather than the diffusion Transformer paradigm, with a StepAudio Tokenizer representing audio at 12.5 Hz in a shared 16x2048 residual code space. Key design principles include interference-aware progressive pretraining, an RVQ Adaptor for multi-codebook acoustic representations, and shared discrete autoregressive modeling. The model reports state-of-the-art performance on TTS and voice design while retaining strong generation across speech, vocals, sound effects, and music.

Hugging Face daily papers·Sep 10, 20:00 UTC · 5d agoAI research

30

DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination

DeCAL, a contact-aware dexterous vision-language-action model with visuo-tactile fusion, reports 71% average task success.

DeCAL is a physically-grounded dexterous vision-language-action (VLA) model built on a Mixture-of-Transformers architecture with specialized experts for understanding, imagination, and action generation. It introduces Adaptive Visuo-Tactile Fusion with contact-aware gating and Visuo-Tactile Latent Co-Imagination to jointly model visual and tactile dynamics. It reports state-of-the-art results with a 71% average success rate and 83.4% progress success rate, plus generalization to unseen scenarios.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:47 UTC · 7d agoAI research1

30

Programmable World Model

Programmable World Model decouples executable world-state evolution from video generation, reaching 94% Count Accuracy and 98% State Accuracy on new CombatStateBench.

An agent translates natural-language instructions into executable programs specifying entity states and transition rules, executed by a lightweight engine that maintains an explicit, persistent global world state including off-screen entities. State-augmented 3D oriented bounding boxes are deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model acting as the generative renderer. On the new CombatStateBench benchmark it achieves 94% Count Accuracy and 98% State Accuracy, substantially outperforming existing interactive video world models.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

27

SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators

SyncWorld is an action-conditioned world model acting as a zero-shot robotics simulator across unseen environments via visual calibration.

Researchers propose SyncWorld, an action-conditioned world model that simulates robot action outcomes in unseen environments without additional training. It uses a visual calibration episode of paired frames and actions to establish the setup-specific Action-Visual Mapping in context. Experiments show accurate simulation of action outcomes in novel settings and that simulated rollouts enable test-time policy improvement without training.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

25

Memory as Plans: World-Action Modeling with Memory-Grounded Planning

Researchers introduce MaP-WAM, decomposing memory-dependent robot manipulation into memory-grounded planning and plan-conditioned execution, achieving 83.3% on RMBench and 78% on real robots.

MaP-WAM converts long-term multimodal episodic memory — segment records with language instructions and sparse visual context — into compact plans of next-segment language goals and visual guidance. A World-Action-Progress model jointly predicts action chunks and execution progress, calibrating predictions via plan-observation alignment for adaptive segment transitions and closed-loop context updates. Structured attention keeps the executor context length fixed and enables key-value caching, yielding state-of-the-art 83.3% success on RMBench, 78.0% on real-robot tasks, and roughly constant inference latency as task history grows.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

32

Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization

SG-JEPA world model conditions latent prediction on physical parameters, halving open-loop prediction error versus DINO-WM and boosting robotic control success.

SG-JEPA extends the LeWorldModel JEPA framework by supplying the governing physics parameter to the temporal model via action-conditioning and jointly training an encoder and predictor through autoregressive latent rollout. On out-of-distribution gravitational-field tasks it reduces open-loop prediction error by up to 2x versus DINO-WM on 2D datasets and increases 3D robotic control success rate up to 2.5x using independently trained diffusion policies. A linear feature analysis attributes most of the gain to the encoder learning features that the predictor can carry forward through rollout.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 17:08 UTC · 6d agoAI research

45

Adobe is trying to make its AI generators idiot-proof in Premiere

Adobe adds in-timeline generative media to Premiere, letting editors generate video and audio clips using Firefly, Veo, Runway, Luma, and Kling models.

Adobe's new Generative Media tool lets Premiere editors highlight empty gaps in the timeline and generate context-aware, editable video, sound effects, music, and soundscapes without leaving the project. Editors can choose among underlying models including Adobe Firefly, Google Veo, Runway, Luma, and Kling. Beta AI audio tools can separate overlapping speakers and duck music under speech, and an AI Assistant is coming to After Effects for plain-language project commands.

The Verge · AI·Sep 8, 13:00 UTC · 7d agoAI industry

22