Search: “action-recognition”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Model

Dynin-Robotics omnimodal diffusion vision-language-action model unifies action, dynamics, and goal prediction, reaching 78.4% success on Franka tasks.

Dynin-Robotics builds a shared trajectory model on the Dynin-Omni omnimodal masked-diffusion backbone, representing language, observations, goals, and actions as discrete tokens. One model learns action prediction, action-conditioned next-observation prediction, terminal goal-state prediction, and trajectory-to-instruction reconstruction, enabling test-time scaling through goal prediction and joint refinement. It is continually pretrained on approximately 1.33 million trajectories from 48 Open X-Embodiment datasets and achieves competitive performance on LIBERO and zero-shot LIBERO-Plus plus a 78.4% average success rate across four manipulation conditions on a Franka Research 3 robot. An optimized block-parallel implementation accelerates model-side action decoding by up to 29.2x.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 16:49 UTC · 4d agoAI research 2 sources1

35

Label-Guided Knowledge Distillation for 3D-CNNs in Action Recognition

LGKD uses ground-truth labels to guide feature distillation for 3D-CNNs, combining sample-wise and class-wise distillation for action recognition.

The paper proposes Label-Guided Knowledge Distillation (LGKD) for 3D-CNNs, noting that most video feature distillation methods are simple adaptations of image techniques that neglect temporal-dimension differences. LGKD combines sample-wise distillation, which uses label information and the teacher's probability distribution to guide features impacting temporal accuracy, with class-wise distillation employing a prototype network to capture relational knowledge among same-category samples. Experiments on the UCF101 and HMDB51 action recognition benchmarks achieve competitive results.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 16:21 UTC · 4d agoAI research

15

Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models

Latent Interface Training improves robot foundation model generalization by constraining visual conditioning, boosting LIBERO-Plus success up to 10.7 points.

The paper identifies vision-action shortcuts where robot policies exploit task-irrelevant visual cues that fail under distribution shift. Latent Interface Training (LIT) first trains an action expert conditioned on language, robot state, and terminal SE(3) end-effector poses without images, then constrains visual input through a pose-supervised latent interface. Across four VLA and world-action architectures (Pi0.5, MolmoAct2, FAST-WAM, ImageWAM), LIT improves LIBERO-Plus success by 3.87-10.70 percentage points. Real-world tests show 13.30-16.70 percentage-point gains under unseen cameras, lighting, and distractors.

Hugging Face daily papers·Sep 10, 20:00 UTC · 5d agoAI research

32

ActionSplice: In-Flight Action Editing for Interactive World Models

ActionSplice enables in-flight action editing in chunk-autoregressive video world models via a lightweight corrector, avoiding rollback or waiting for the next chunk.

ActionSplice is an inference framework that formulates in-flight action editing for chunk-autoregressive video world models as Counterfactual State Transport (CST), where a lightweight corrector transports the interrupted backbone-native representation toward the matched state induced by the revised action. The world model and sampler remain frozen, and sampling resumes without replaying completed evaluations. Across minWM-Wan Action2V and HY-WM1.5, the retargeting variant CST-R reduces rollback-relative LPIPS by 61.5% and 75.9% versus direct condition swapping, while the temporal-splicing variant CST-T reduces suffix LPIPS by 56.1% and 77.5% with 2.73x and 1.69x pixel-ready speedups over waiting.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

22

What Matters, When? Diagnosing and Improving Conditional Visual Grounding in Visuomotor Imitation Policies

Researchers diagnose conditional visual grounding failures in visuomotor imitation policies and show targeted interventions substantially improve distractor robustness.

The paper studies why ACT-based visuomotor imitation policies fail when visually similar distractor objects or receptacles are introduced, finding sensitivity depends on both distractor type and manipulation stage. Interventions including distractor augmentation, phase-dependent attention regularization, and appearance-based visual prompting improve target selection while preserving spatial control information, with gains in simulation and on a physical UR3e. The same failure pattern is confirmed in a pretrained vision-language-action policy on a state-conditioned medical instrument-handling task.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:27 UTC · 11d agoAI research

18

SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Models

SimpleMemVLA passes full timestamped video history straight to a VLA backbone, setting state of the art on four memory benchmarks.

SimpleMemVLA is a vision-language-action model for long-horizon manipulation that removes the dedicated memory module entirely. It keeps sampled history intact and feeds it to the backbone as timestamped video, with the hidden states of a generated sub-task serving as the only channel into a standard flow-matching action head. Prefilling the shared history prefix during action execution keeps latency close to a single-frame VLA. The system sets a new state of the art on four memory benchmarks and outperforms retrieval, compression and recurrent-state mechanisms, with causal interventions confirming the policy genuinely reads its history.

Hugging Face daily papers·Sep 1, 20:00 UTC · 14d agoAI research

25

PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models

PhysBrain 1.5, an 8B physical foundation model, sets open-source state of the art across 28 embodied understanding benchmarks.

The paper presents PhysBrain 1.5, a unified 8B model for understanding physical environments, generating actions, and predicting future states, built from a vision-language model with joint autoregressive next-token prediction over language, end-effector motion, and dense visual targets. Pre-training uses embodied supervision from human interaction videos, followed by supervised fine-tuning on human demonstrations, robot trajectories, and simulated experience. The model averages 72.5 across 28 embodied benchmarks, setting a new open-source state of the art and performing on par with proprietary GPT-6-Astra and Gemini 3.6 Flash, with best open-source results on 14 benchmarks.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI research1

45

OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining

OpenWAM releases an open modular stack for world-action model pretraining, plus OpenWAM-alpha trained on about 6,400 hours of egocentric and robot data.

OpenWAM is an open research stack that factorizes World-Action Model pretraining into composable infrastructure, study, and model components with unified training, inference, and evaluation. Controlled experiments distill three principles on knowledge inheritance, world-action synergy, and out-of-domain generalization gains from embodied co-training. The resulting OpenWAM-alpha, pretrained on roughly 6,400 hours of egocentric human and robot data, achieves top-tier results across eight simulation benchmarks and real-robot tests spanning single-arm, bimanual, and dexterous embodiments. The full stack, including infrastructure, evaluation protocols, pretrained models, and data recipes, is released openly.

Hugging Face daily papers·Sep 6, 20:00 UTC · 9d agoAI research

44

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

Researchers introduce EmbodiedSkills, a framework treating VLA skill decisions as verified execution proposals, reaching 86.2% success on RoboTwin 2.0.

The EmbodiedSkills framework treats each vision-language-action skill decision as an execution proposal, checking prerequisites before execution and verifying outcomes afterward via a shared executable-skill interface. It connects high-level skill selection, bounded low-level VLA execution and post-action verification in a single agent loop, and logs structured trajectories for supervision and optional online adaptation. Instantiated with Qwen3-VL and OpenPI/pi0.5, task-adapted policies achieve 86.20% average success across 50 RoboTwin 2.0 tasks and 97.40% across the four LIBERO suites, with 12.5% on memory-dependent RMBench tasks.

Hugging Face daily papers·Aug 31, 20:00 UTC · 15d agoAI research1

46

GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

GE-Act 2.0 is a from-scratch pretrained world-action model for robotic manipulation, with success rising from 17.1% to 44.1% as co-training data scales to 30,000 hours.

Genie Envisioner Act 2.0 (GE-Act 2.0) is a world-action model whose generative and action components are all initialized from scratch on manipulation data, combining a control-oriented autoencoder (CoAE), single-step visual planner (SVP), and inverse dynamics model (IDM) trained jointly via knowledge-aligned selective optimization (KASO). Scaling co-training data from 300 to 30,000 hours raises zero-shot success from 17.1% to 44.1% on G1-OP and 13.4% to 31.1% on G2-90D, despite the latter comprising under 2% of data, suggesting cross-embodiment transfer. Gains span 19/20 and 18/20 skill groups, and skill-specific coverage correlates with zero-shot OOD success (Pearson r=0.80).

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research

30

Memory as Plans: World-Action Modeling with Memory-Grounded Planning

Researchers introduce MaP-WAM, decomposing memory-dependent robot manipulation into memory-grounded planning and plan-conditioned execution, achieving 83.3% on RMBench and 78% on real robots.

MaP-WAM converts long-term multimodal episodic memory — segment records with language instructions and sparse visual context — into compact plans of next-segment language goals and visual guidance. A World-Action-Progress model jointly predicts action chunks and execution progress, calibrating predictions via plan-observation alignment for adaptive segment transitions and closed-loop context updates. Structured attention keeps the executor context length fixed and enables key-value caching, yielding state-of-the-art 83.3% success on RMBench, 78.0% on real-robot tasks, and roughly constant inference latency as task history grows.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

32

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled pure-autoregressive testbed shows task-specific validation losses rank image tokenizers differently, with I2T loss the most consistent signal.

Researchers built a controlled pure-autoregressive testbed and tracked task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They find losses should be analyzed per task because they exhibit distinct scaling behavior and rank tokenizers differently, and that the loss-performance relationship depends on the predicted token space. I2T loss, computed over a shared text vocabulary, correlates consistently with both generation and visual understanding performance after supervised finetuning. Case studies revisit the discriminator, semantic supervision, and vocabulary size as tokenizer design axes.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:57 UTC · 7d agoAI research1

30

Show-Harness: Just a VLM Agent Can Play Robots

Show-Harness lets VLM agents control robots via discrete semantic action units, outperforming VLA baselines zero-shot and after light fine-tuning.

Show-Harness is an embodied agent harness that exposes discrete semantic action units a VLM reasons over, with embodiment-specific interpreters grounding them into local robot actions. It enables zero-shot robot control with closed-source frontier VLMs and low-cost adaptation of small open-source VLMs using only a few GPU-hours of fine-tuning. The companion GUMI (GUI Manipulation Interface) extends the same semantic action space to GUI-based demonstration collection without specialized teleoperation hardware. Experiments show robust generalization across tasks, embodiments, and environments, beating representative agentic and VLA paradigms.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 17:53 UTC · 6d agoAI research

32

DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination

DeCAL, a contact-aware dexterous vision-language-action model with visuo-tactile fusion, reports 71% average task success.

DeCAL is a physically-grounded dexterous vision-language-action (VLA) model built on a Mixture-of-Transformers architecture with specialized experts for understanding, imagination, and action generation. It introduces Adaptive Visuo-Tactile Fusion with contact-aware gating and Visuo-Tactile Latent Co-Imagination to jointly model visual and tactile dynamics. It reports state-of-the-art results with a 71% average success rate and 83.4% progress success rate, plus generalization to unseen scenarios.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:47 UTC · 7d agoAI research1

30

Reward AI Releases OM-1: A Robot Policy Trained on Human Demonstrations Only, With No Teleoperation or On-Robot Data

Reward AI released OM-1, a general-purpose manipulation policy trained solely on human demonstrations from a sensorized glove, with no teleoperation or robot data.

Reward AI announced OM-1 (Omnibody Model 1), a general-purpose robot manipulation policy trained only on human demonstrations captured via Omnibody Hand, a 7-DoF wearable glove with tactile, proximity, and in-hand camera sensing. The system uses electromagnetic hand-pose tracking, cutting mean overshoot error to 9.5 mm versus 24.9 mm for visual-inertial at 67 cm/s (a 60% reduction), and reportedly learns brand-new tasks from under 30 minutes of human data. A separate RL-trained control layer runs on its own clock so policy inference latency never stalls motion, and the policy spans industrial arms, legged humanoids, and wheeled mobile manipulators. No weights, code, dataset, API, paper, or benchmark comparisons have been released, so claims are demonstration-backed only.

MarkTechPost·Sep 14, 21:08 UTC · 1d agoAI research

35

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled autoregressive testbed shows validation losses must be analyzed per task, and image tokenizer choice affects joint multimodal text modeling.

Researchers built a pure-autoregressive testbed to study image tokenizers as the 'visual language' of unified multimodal models, tracking task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They found that losses exhibit distinct scaling behavior per task and rank tokenizers differently, and that I2T loss over a shared text vocabulary gives a more consistent loss–performance signal than T2I loss. Better reconstruction does not necessarily yield lower task-specific losses or stronger downstream performance, and tokenizer choice can affect text modeling under joint optimization. Case studies examine the discriminator, semantic supervision, and vocabulary size design axes.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research1

40

Ambient @ EgoProactive 2026 : Proactive Egocentric Assistance with Visually Grounded Supervision

ECCV 2026 challenge winner reformulates egocentric intervention timing as single-token classification, boosting macro-F1 by 0.249 over free-form generation.

The paper describes the winning submission to the EgoProactive track of the ECCV 2026 Wearable AI Challenge, ranking first in the large-model division and second in the <=2B division. The method reformulates intervention timing as single-token yes/no classification, improving macro-F1 by 0.249 and G-mean by 0.30 over free-form generation. Supervision generated by a tool-calling video agent transferred better than a narration-only dataset that was four times larger and ten times cheaper, suggesting visual grounding matters more than annotation volume.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

22

Same Trajectory, Contradictory Rewards (ROBORMBENCH): Paraphrase Fragility in Vision Language Reward Models

New ROBORMBENCH benchmark shows vision-language reward models can flip robot success/failure judgments when goal instructions are paraphrased.

The authors show that paraphrasing the instruction alone can substantially change progress scores from VLM reward models, even flipping identical robot trajectories between failure and success. ROBORMBENCH comprises 2,390 real-robot trajectories with ground-truth progress labels and 21,673 verified paraphrases covering lexical, syntactic, and action-goal rewrites. Instability is widespread across proprietary and open-source VLMs, grows with more divergent rewrites, and is not reliably reduced by scale or explicit reasoning, while trajectory-grounded dedicated reward models are markedly more stable.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:47 UTC · 11d agoAI research1

35

MobileVLA-R1 2.0: RL-Enhanced Reasoning for Mobile Robot Control

MobileVLA-R1 2.0 couples chain-of-thought reasoning with RL for mobile robot control, gaining 10 points on real Unitree G1 tasks.

MobileVLA-R1 2.0 is an RL-enhanced vision-language-action framework that explicitly couples structured embodied reasoning with executable mobile robot control via supervised Chain-of-Thought alignment and reinforcement learning. A reasoning-conditioned action decoder maps multimodal reasoning representations to task-level action targets, decoupling high-level action generation from robot-specific actuation for both locomotion and manipulation. It achieves an average 1.6 point SR improvement on VLN-CE and a 10.0 point improvement in full-task success on real-world Unitree G1 mobile manipulation, with evaluations covering navigation, quadruped control, and real deployments on Unitree Go2 and G1 robots.

Hugging Face daily papers·Sep 4, 20:00 UTC · 11d agoAI research

32

TrajMark: Ownership Attribution and Segment-Level Tamper Localization for Coding-Agent Trajectories

Researchers introduce TrajMark, a training-free watermarking framework for coding-agent trajectories that recovers ownership, detects 95.5-100% of edits, and localizes tampered regions.

TrajMark is a training-free, symmetric-key, visible-only watermarking framework for coding-agent trajectories that separates robust ownership attribution from fragile local integrity verification. A sparse owner layer encodes a six-bit deployment identifier by rewriting keyed READ actions into masked linear equations, while a localization layer inserts linked Q12 seals that commit to protected critical-action segments. Across three coding-agent frameworks and three LLMs, it recovers the exact owner in all clean full-watermark batches, detects 95.5%-100% of single-site edits, and localizes 95.8% of random corruptions to an accepted protocol region. Owner marking adds no trajectory actions and matched Pass@1 is 26.9% versus 26.3% for unwatermarked runs.

arXiv cs.CR·Sep 9, 16:33 UTC · 6d agoResearch1

30

Do Input-Level Defenses Transfer to Observation-Level Attacks on VideoLLMs?

A systematic study shows input-level adversarial defenses provide inconsistent, often near-zero protection against observation-level attacks on video LLMs.

Researchers introduce DefTEval, a controlled framework testing eleven input-level defenses against five attack types across five video LLMs. Harmful-content detection rates are frequently near zero, and defenses fail even when attacks embed harmful signals in every sampled frame. Token compression discards localized safety features and modality fusion down-weights weakened visual signals, with defense outcomes dominated by model architecture rather than the defense method.

arXiv cs.CR·Sep 8, 06:59 UTC · 8d agoAI safety & security

40

TANGO: Humanoid Navigation in Cluttered Environments with a Whole-Body Vision-Language-Action Model

Researchers introduce TANGO, a whole-body vision-language-action model enabling zero-shot language-guided humanoid navigation on the Unitree G1 robot.

TANGO addresses humanoid navigation in cluttered indoor environments by predicting 29-DoF joint-space actions directly from natural-language instructions and egocentric RGB, rather than 2D path planning. It is trained entirely in simulation via a pipeline combining global path planning, kinematic whole-body motion generation, obstacle-aware motion editing, and RL-based tracking. In simulation it achieves state-of-the-art vision-language navigation performance and transfers zero-shot to a Unitree G1 humanoid without any real-world navigation data.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research1

27

LLM Forensics: Where Do Backdoors Hide? Localizing and Controlling Trigger Mechanisms with Sparse Autoencoders

Researchers use sparse autoencoders to localize trigger-based backdoor mechanisms in 1B and 8B LLMs, finding detection features differ from causal control features.

In a controlled language-switching backdoor setting where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German, the authors train sparse autoencoders (SAEs) across layers and transformer components. Attention and MLP features detect triggered prompts with near-perfect F1, but ablating them rarely suppresses the language switch, while residual-stream features can suppress triggered generation and some can induce target-language continuations without the trigger. The work decomposes token-trigger mechanisms into distinct SAE feature roles: trigger detection, residual-stream propagation, and language tracking, a decomposition the authors expect to transfer to other trigger-based backdoors.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 16:47 UTC · 8d agoAI safety & security1

55

Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

Researchers introduce Motion-Omni, an end-to-end model generating speech with synchronized full-body motion, responding 5.4x faster than cascade pipelines.

Motion-Omni is an end-to-end framework in which a spoken dialogue model outputs facial expressions and hand, upper-body, and lower-body motion directly from the hidden states that produce speech, replacing two-stage cascade pipelines. Trained on 422,856 quality-ranked pseudo-labeled pairs (1,402 hours) with a Qwen2.5-7B-Instruct backbone, Motion-Omni-Q7 matches its teacher cascade within 2% on reference-free motion metrics, achieves a 2.62% word error rate, and runs faster than real time (RTF=0.78). The authors also release the SwDA-500 dataset and the first public evaluation protocol for stochastic open-ended full-body spoken dialogue.

Hugging Face daily papers·Aug 27, 20:00 UTC · 19d agoAI research1

45

On Identifying Adversarial Intent Injection in AI-Native 6G Networks

Dual-path CNN and AutoEncoder framework detects adversarial intent injection in AI-native 6G networks, reaching 0.97 accuracy and 0.98 F1.

The paper defines a fine-grained threat model for adversarial intent injection in AI-native 6G intent-based networking, where malicious policies are disguised within benign intent flows. It evaluates four injection strategies: stealth-mode, random distribution, increasing frequency, and decreasing frequency. A dual-path detection framework combines a CNN using TF-IDF features for supervised detection with an AutoEncoder trained only on benign data for one-class detection, reaching 0.97 accuracy and 0.98 F1-score, roughly 9% and 36% gains over the state-of-the-art baseline.

arXiv cs.CR·Sep 10, 19:23 UTC · 5d agoResearch

35

A Chosen Future Can Still Be Rewritten: Causal Writability in Video Models

Study shows video models often learn correct physics but fail to use it; low-dimensional 'causal writability' edits can restore correct motion.

The paper demonstrates 'causal writability' in video generation models: physically correct motion remains available inside the model even when the model outputs incorrect motion. In a red/blue mass oscillation setup, a low-dimensional edit predicted from simple physical variables restores correct fast motion, with a sharp depth boundary marking commitment. Early causal writability predicts which training errors later get corrected, and both writability and closure reproduce in a pretrained 1.3B video model.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:58 UTC · 1d agoAI research

33

Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy

Researchers added Greek to the Cosmos3 vision-language-action policy using only machine-rephrased instructions, finding bilingual training reaches roughly two fifths of English performance.

The paper studies localizing the open Cosmos3 vision-language-action robot policy to Greek without architectural changes, using machine-rephrased instructions only. Bilingual training yields a consistent 6.7-7.1 point margin over controls on a 90-task, three-seed evaluation suite, while Greek-only training gains at most 2.7 points. Several common evaluation instruments, including color-histogram metrics and single-goal benchmarks, produced false conclusions, and results were dominated by seed variation. The authors recommend building guaranteed-null baselines and replicating low-resource-language results across seeds.

Hugging Face daily papers·Sep 6, 20:00 UTC · 9d agoAI research

24

Show-Harness: Just a VLM Agent Can Play Robots

Show-Harness enables VLM agents to control robots via a semantic action interface, achieving zero-shot frontier control and few-GPU-hour adaptation of small VLMs.

Show-Harness exposes discrete semantic action units that VLMs reason over, with embodiment-specific interpreters deterministically grounding them into local robot actions. It enables zero-shot closed-source frontier VLM control and adapts small open-source VLMs for low-cost deployment with a few GPU-hours of fine-tuning. The companion GUMI interface extends the same semantic action space to GUI-based demonstration collection without teleoperation hardware, and Show-Harness-equipped agents outperform representative agentic and VLA paradigms.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

28

RoboSPA: Can VLA Models Go Beyond Simple Scenes and Short-Horizon Tasks?

RoboSPA introduces a 527K-trajectory robotic benchmark with 280 task variants exposing VLA model weaknesses in spatial reasoning and long-horizon planning.

RoboSPA is a large-scale robotic manipulation dataset and benchmark for diagnosing embodied reasoning in vision-language-action models, spanning fine-grained spatial reasoning and long-horizon procedural planning. It covers 10 task categories and 56 base tasks instantiated across five difficulty levels for 280 variants, with 527K trajectories collected across multiple embodiments. Experiments on representative VLA models show current systems struggle with complex spatial relations, precise low-level execution, and memory-intensive planning.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:19 UTC · 11d agoAI research

25