Search: “performance-capture”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control

PhysStream autoregressive video model enables physics-grounded mid-generation motion control, cutting trajectory error 12% and FVMD 33% versus strongest baselines.

PhysStream is an autoregressive image-to-video model that incorporates structured scene memory—positional maps and object tracking maps derived online from previously generated frames—and supports fine-grained motion control via sparse velocity-increment signals encoding physical quantities. Training proceeds in two stages: a bidirectional model finetuned with motion-control conditioning, then a causal autoregressive model with scene memory. It reduces motion distribution distance (FVMD) by 33% and trajectory error by 12% over the strongest baselines, and human evaluators prefer it in over 85% of in-the-wild comparisons.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:55 UTC · 21h agoAI research 2 sources

25

TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents

TRACE, a training-free visual token pruning framework, cuts GUI agent inference latency and memory while keeping trajectory-wide visual evidence reusable.

TRACE is a training-free framework for trajectory-robust admission and coverage-aware evidence ordering that prunes high-resolution screenshot tokens accumulated in GUI agent trajectories. It ranks visual evidence using a query-independent layout-derived interaction prior combined with instruction relevance and feature novelty, and reserves part of the budget for native tokens distributed across the screen to repair spatial coverage. A monotone KV contraction incrementally compresses retired frames into compact session state, avoiding repeated visual encoding or pruning. Experiments across six GUI benchmarks and diverse models verify effectiveness under tight budgets, with source code to be released.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

28

Measuring benchmark optimization in speech recognition

Hugging Face examines how much speech recognition systems overfit benchmarks and how to measure benchmark optimization in ASR.

A Hugging Face post on measuring benchmark optimization in automatic speech recognition, analyzing how model improvements on benchmarks reflect genuine capability gains versus overfitting. It is evaluation methodology research with no direct security impact.

Hugging Face Blog·Aug 21, 00:00 UTC · 26d agoAI research

22

TrajMark: Ownership Attribution and Segment-Level Tamper Localization for Coding-Agent Trajectories

Researchers introduce TrajMark, a training-free watermarking framework for coding-agent trajectories that recovers ownership, detects 95.5-100% of edits, and localizes tampered regions.

TrajMark is a training-free, symmetric-key, visible-only watermarking framework for coding-agent trajectories that separates robust ownership attribution from fragile local integrity verification. A sparse owner layer encodes a six-bit deployment identifier by rewriting keyed READ actions into masked linear equations, while a localization layer inserts linked Q12 seals that commit to protected critical-action segments. Across three coding-agent frameworks and three LLMs, it recovers the exact owner in all clean full-watermark batches, detects 95.5%-100% of single-site edits, and localizes 95.8% of random corruptions to an accepted protocol region. Owner marking adds no trajectory actions and matched Pass@1 is 26.9% versus 26.3% for unwatermarked runs.

arXiv cs.CR·Sep 9, 16:33 UTC · 6d agoResearch1

30

ECHO: A Matched-Contrast Benchmark for Context-Sensitive Turn-Taking in Full-Duplex Dialogue

ECHO benchmark pairs identical-overlap Chinese dialogue examples with contrasting contexts, revealing most full-duplex systems bias toward yielding the floor.

ECHO is a paired diagnostic benchmark for Chinese full-duplex turn-taking that matches examples with identical overlap transcripts but contrasting preceding multi-turn contexts, one requiring Yield and the other Keep. It also includes off-talk examples for diagnosing unnecessary yielding and introduces pair accuracy, which grants no credit to constant-action policies. Experiments across multiple full-duplex systems show most exhibit a pronounced Yield bias, performing substantially better on interruptions than backchannels. The benchmark and metadata will be publicly released.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 15:58 UTC · 23h agoAI research

40

What Matters, When? Diagnosing and Improving Conditional Visual Grounding in Visuomotor Imitation Policies

Researchers diagnose conditional visual grounding failures in visuomotor imitation policies and show targeted interventions substantially improve distractor robustness.

The paper studies why ACT-based visuomotor imitation policies fail when visually similar distractor objects or receptacles are introduced, finding sensitivity depends on both distractor type and manipulation stage. Interventions including distractor augmentation, phase-dependent attention regularization, and appearance-based visual prompting improve target selection while preserving spatial control information, with gains in simulation and on a physical UR3e. The same failure pattern is confirmed in a pretrained vision-language-action policy on a state-conditioned medical instrument-handling task.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:27 UTC · 11d agoAI research

18

Domain-Incremental Learning for Multi-Channel Replay Speech Detection

First continual learning benchmark for multi-channel replay speech detection shows task-specific beamforming cuts catastrophic forgetting across 24 acoustic environments.

Researchers frame replay-attack detection for voice-controlled systems as domain-incremental learning over acoustic environments, evaluating a beamformer-based detector across all 24 environment orderings of the ReMASC corpus with five seeds. Naive sequential fine-tuning raises error rates on previously learned environments by 18.8 points, while elastic weight consolidation halves forgetting but loses plasticity and gradient projection memory is statistically indistinguishable from naive fine-tuning. A task-specific beamformer keeping one spatial front-end per environment significantly improves final and incremental accuracy, and the last environment in a sequence dominates final performance.

arXiv cs.CR·Sep 10, 08:03 UTC · 6d agoResearch1

22

ActionSplice: In-Flight Action Editing for Interactive World Models

ActionSplice enables in-flight action editing in chunk-autoregressive video world models via a lightweight corrector, avoiding rollback or waiting for the next chunk.

ActionSplice is an inference framework that formulates in-flight action editing for chunk-autoregressive video world models as Counterfactual State Transport (CST), where a lightweight corrector transports the interrupted backbone-native representation toward the matched state induced by the revised action. The world model and sampler remain frozen, and sampling resumes without replaying completed evaluations. Across minWM-Wan Action2V and HY-WM1.5, the retargeting variant CST-R reduces rollback-relative LPIPS by 61.5% and 75.9% versus direct condition swapping, while the temporal-splicing variant CST-T reduces suffix LPIPS by 56.1% and 77.5% with 2.73x and 1.69x pixel-ready speedups over waiting.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

22

Feature Recovery for Object Understanding After Irreversible Fire Damage

TRACE benchmark with 21.4K scenes studies post-fire object understanding; a Feature Recovery Module improves degraded-image retrieval by 12.5% and material recovery by 20.1%.

The paper introduces TRACE, a transformation-aware benchmark with 21.4K real-image-grounded synthetic scenes, 499 object identities across 189 categories, and five tasks covering degraded-object detection, pristine-state recovery, material recovery, description generation, and functional reasoning. Existing models degrade sharply: RF-DETR mAP falls 71% relative from least to most severe level, and InternVL3.5 retrieval R@1 drops from 93.85 to 28.11. The proposed Feature Recovery Module maps degraded encoder features to pristine-aligned representations while keeping the host model frozen, averaging relative gains of 12.5% for retrieval and 20.1% for material recovery across VLM hosts and severity levels.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

30

SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Models

SimpleMemVLA passes full timestamped video history straight to a VLA backbone, setting state of the art on four memory benchmarks.

SimpleMemVLA is a vision-language-action model for long-horizon manipulation that removes the dedicated memory module entirely. It keeps sampled history intact and feeds it to the backbone as timestamped video, with the hidden states of a generated sub-task serving as the only channel into a standard flow-matching action head. Prefilling the shared history prefix during action execution keeps latency close to a single-frame VLA. The system sets a new state of the art on four memory benchmarks and outperforms retrieval, compression and recurrent-state mechanisms, with causal interventions confirming the policy genuinely reads its history.

Hugging Face daily papers·Sep 1, 20:00 UTC · 14d agoAI research

25

Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs

A survey catalogs inference-efficiency techniques for video and audiovisual LLMs, mapping bottlenecks in sampling, encoding, token reduction, and LLM decoding.

This survey covers inference-efficiency mechanisms for visual and audiovisual video LLMs, reporting reductions in parameters, FLOPs, latency, memory, and token counts. It organizes methods by pipeline stage, covering frame sampling, modality encoding, connector-level token reduction, and LLM prefilling and decoding for systems built since late 2022. The authors compile accuracy-cost comparisons under shared host models and input protocols, identify gaps in audiovisual efficiency and standardized evaluation, and maintain a public repository.

Hugging Face daily papers·Sep 8, 20:00 UTC · 7d agoAI research

40

HazardAuditor: From Executable Threats to Safer Computer-Use Agents

HazardAuditor trains execution-grounded guard models for computer-use agents, improving safety verdict accuracy by up to 16.5 points.

HazardAuditor runs heterogeneous agents (Claude Code, Codex, Hermes, OpenClaw) in controlled environments and normalizes their interactions into a canonical event representation for cross-framework supervision. It introduces Guard Policy Optimization (GuardPO), which converts deterministic safety outcomes into sequence-level advantages and normalizes rationale and verdict regions so the safety decision becomes the effective optimization unit. Across multiple benchmarks and heterogeneous computer-use systems, HazardAuditor improves accuracy by up to 16.5 percentage points over the strongest prior guard model. Code, models, and evaluation artifacts are being released.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI safety & security

55

Continue, Adapt, or Yield: In-Turn Adaptation to Overlapping Speech in Full-Duplex Agents

Duplex Cue evaluation shows PersonaPlex full-duplex agents adapt in-turn to listener contributions in only 34.8% of collaborative cases versus 68.2% for humans.

The paper introduces Duplex Cue, an evaluation of in-turn adaptation in full-duplex voice agents that separates listener intent (backchannel, collaboration, interruption) from speaker behavior (continue, adapt, yield). Using 208 scorable pairs from 300 human-confirmed cues in unscripted English conversations, it compares recorded human responses with PersonaPlex continuations generated while replaying listener audio. Humans adapt within the turn in 68.2% of collaborative pairs versus 34.8% for PersonaPlex, which otherwise continues unchanged (42.4%) or yields (22.7%).

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:46 UTC · 4d agoAI research

15

BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure

BenchShield uses lifecycle-model-backed instrumentation to detect reward hacking in LLM-agent benchmarks, lifting full-chain recall to 77-100% at up to 65% lower cost.

The framework grounds reward-hacking detection in a finite lifecycle model of an evaluation's reward-relevant events, combining a static phase-aware taint analysis with runtime infrastructure-side evidence attribution. Evaluation used a human-labeled corpus of 456 adjudicated trajectories drawn from more than 31,000 public agent runs across three benchmarks. BenchShield improves full-chain recall from 23-94% to 77-100% and same-vector coverage from 16-56% to 43-78%, cuts per-task cost by up to 65%, and achieves 96% accuracy detecting reward hacking at runtime.

arXiv cs.CR·Sep 10, 03:10 UTC · 6d agoAI safety & security1

42

MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant

MP-Bench is the first benchmark for voice agents in multiparty conversations, finding real-time agents near chance on turn-taking.

MP-Bench is the first benchmark designed to objectively evaluate conversational speech systems as active participants in multi-party conversations. It assesses agents on turn-taking awareness and response appropriateness, with comprehension-based question-answering as a complementary evaluation. Benchmarking 12 voice agents shows real-time agents score at or below 22% on multiparty comprehension and remain near chance on multiparty turn-taking.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:11 UTC · 4d agoAI research2

20

Design Docs Are All You Need: An AI-native Machine-Learning Performance Tool

Researchers present SMART, an ML performance-modeling library regenerated by AI coding agents from natural-language design docs instead of code.

The paper describes SMART, a symbolic performance-modeling library whose main branch contains almost no code: the repository is a DAG of self-contained design documents, and coding sub-agents regenerate implementations from only the docs on version updates. Reliability rests on a worked-example doc style used as in-context demonstrations and a minimal operator IR with SymPy cost expressions, offering both fast analytical roll-up and fine-grained modulo-scheduling modes. Regenerated implementations reproduce hand-audited reference models, including DeepSeek-V3 serving on a TPU pod slice, to round-off precision.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:08 UTC · 11d agoAI research1

30

DRIFT: Removing Diffusion Watermarks by Deflecting the Generative Trajectory

Introduces DRIFT, a black-box attack removing diffusion watermarks by deflecting generative trajectories, achieving 98-100% success across nine watermarking schemes.

Researchers propose DRIFT, a black-box watermark removal attack combining partial forward diffusion with stochastic reverse resampling to break trajectory-dependent verification. The paper derives information-theoretic and Wasserstein source-dependence bounds and shows the first verifier-rejected rung is least distorted among rejected rungs. Across nine watermarks spanning three paradigms, DRIFT achieves 98-100% attack success with the best image quality among compared attacks, without secret keys, verifier internals, or per-image gradient optimization.

arXiv cs.CR·Sep 8, 03:54 UTC · 8d agoResearch

38

Omni-Streaming Thinking

Omni-Streaming Thinking fixes premature cross-modal commitment in streaming omni-modal models via pending claims verified against modality-specific evidence, beating baselines by over 10%.

The paper identifies 'premature cross-modal commitment', where streaming models keep relaying early visual interpretations even after audio contradicts them. OST generates evidence-linked pending claims with future verification intervals, stores audio and visual evidence separately, and refutes claims when contradictory evidence appears. Built on a frozen Qwen3-Omni-30B-A3B-Instruct backbone with lightweight adaptation, it outperforms open baselines by more than 10% relative on five streaming and audio-visual benchmarks. On the new OST-DiagBench it reaches d-prime 2.95 versus at most 1.38 for open baselines, while reducing vision-induced auditory hallucinations.

Hugging Face daily papers·Sep 13, 20:00 UTC · 2d agoAI research1

45

EmbodiedSkills: A Unified Framework for Orchestrating, Training, and Deploying VLA Agents

Researchers introduce EmbodiedSkills, a framework treating VLA skill decisions as verified execution proposals, reaching 86.2% success on RoboTwin 2.0.

The EmbodiedSkills framework treats each vision-language-action skill decision as an execution proposal, checking prerequisites before execution and verifying outcomes afterward via a shared executable-skill interface. It connects high-level skill selection, bounded low-level VLA execution and post-action verification in a single agent loop, and logs structured trajectories for supervision and optional online adaptation. Instantiated with Qwen3-VL and OpenPI/pi0.5, task-adapted policies achieve 86.20% average success across 50 RoboTwin 2.0 tasks and 97.40% across the four LIBERO suites, with 12.5% on memory-dependent RMBench tasks.

Hugging Face daily papers·Aug 31, 20:00 UTC · 15d agoAI research1

46

Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation

Benchmark Radar provides a living searchable database of 1,283 AI benchmark records and 12,916 score observations drawn from 37 daily discovery sources.

Benchmark Radar combines daily discovery of benchmark papers, repositories, datasets, and releases from 13 direct connectors and 24 first-party feeds into a searchable catalog with model card mentions and score histories. The catalog contains 1,283 source records drawn from 4 benchmark catalogs plus 12,916 numeric observations on 790 records. The release includes a web dashboard with leaderboard, Pareto frontier of score versus usage, saturation and trend views, daily feeds, a CLI, and reproducible analysis. The paper audits the full catalog and examines benchmark saturation and limits of score comparisons.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

25

What Did I Just Say? Self-Listening for Full-Duplex Speech Models

Researchers propose Self-Listening, a full-duplex speech approach feeding realized model speech back as input to improve interruption recovery.

Full-duplex spoken language models can listen and speak simultaneously, but asynchronous text generation, speech synthesis, and playback cause mismatches between what a model believes it said and what the user heard. The paper defines the resulting recovery problem as anchor interruption and proposes Self-Listening, which interleaves user speech, model text, and played speech as input streams. The authors also release AnchorSpeech, a benchmark with homogeneous training and test splits tracking which ordered response items were actually spoken. Experiments show self-listening models achieve better anchoring performance than full-duplex baselines.

Hugging Face daily papers·Sep 3, 20:00 UTC · 12d agoAI research

22

Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout

Researchers propose Mask Forcing, a dual-noise masking rollout that mitigates mode collapse in autoregressive video diffusion distillation.

The paper targets over-saturation and over-smoothing in autoregressive video diffusion models distilled via Distribution Matching Distillation, attributed to reverse-KL mode-seeking behavior. Mask Forcing perturbs the student self-rollout with random masks along spatial and temporal axes, injecting cleaner tokens that act as denoising guidance for noisier tokens. Experiments show improved visual quality across multiple distillation methods without using real video data or extra post-training stages.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

22

Environments as Scaffold: Enriching Feedback to Bootstrap Self-Evolving Agents in Long-Horizon Tasks

Researchers propose Feedback-Enriched Environments (FEEs) that reduce reward sparsity and improve RL training of Qwen3-based agents on SciWorld and BFCL.

The paper proposes shifting from agent-side warmup (SFT) to environment-side adaptation via Feedback-Enriched Environments to address severe reward sparsity in RL training of long-horizon LLM agents. A pilot study defines a feedback strategy that transitions from action guidance to observation enrichment in later training stages. Large-scale experiments on SciWorld and BFCL across Qwen3 model scales and GRPO, GSPO, and DAPO show consistent gains, plus stabilized training dynamics and proactive exploration.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

25

FreeFlow: A Bias-free Hierarchical Transformer for Optical Flow Estimation

FreeFlow is a bias-free hierarchical transformer achieving state-of-the-art optical flow results on Sintel, KITTI-2015, and Spring benchmarks.

FreeFlow replaces task-specific inductive biases like correlation volumes and iterative warping with a single feed-forward encoder-decoder combining window, shifted-window, and reduced-resolution global attention. It reaches 0.68/1.48 EPE on Sintel Clean/Final, 3.23 Fl-all on KITTI-2015, and 3.192 1px on Spring. The architecture scales consistently from small to large variants and remains memory efficient at 1080p inference.

Hugging Face daily papers·Sep 9, 20:00 UTC · 6d agoAI research

30

What Makes Adversarial Examples Transfer Across Deepfake Detectors?

A controlled study of 60 deepfake detectors shows adversarial example transfer depends heavily on source-target compatibility, with source averaging understating vulnerability.

The study evaluates adversarial example transferability across 60 deepfake detectors spanning six backbones, two pretraining regimes, and five training-data configurations, using AutoAttack (AA) and Carlini-Wagner with Expectation over Transformation (CW-EOT). Transfer rises sharply when source and target share an exact backbone, architecture family, pretraining regime, or training data, with the dominant factor depending on the attack. Mean attack success rate is 7.21% under AA and 19.52% under CW-EOT for single sources, while a multi-source oracle reaches 64.48% after excluding exact matches, showing source averaging can substantially understate target vulnerability. The authors release 240,000 adversarially perturbed images, pairwise transfer results, detector configurations, and evaluation code.

arXiv cs.CR·Sep 9, 10:28 UTC · 7d agoAI safety & security

40

TempCloze: Can Video-LLMs Identify the Missing Middle?

TempCloze benchmark tests Video-LLMs' temporal reasoning with 1,521 videos, finding temporal alignment is the primary failure mode across 31 models.

TempCloze is a video cloze benchmark in which models must identify the true missing middle clip given the beginning and ending clips, using 1,521 carefully filtered videos from seven sources, mostly long-take and egocentric footage. Distractors are constructed along three dimensions: Semantic, Alignment and Progression, with shared scenes and objects to reduce appearance cues. Evaluation of 10 proprietary and 21 open-source Video-LLMs found Alignment is the primary bottleneck, with models often recognizing plausible semantics and local event progression but struggling with temporal alignment.

Hugging Face daily papers·Aug 31, 20:00 UTC · 15d agoAI research

25

MeClear: Cooperative Game-Theoretic Attribution and Risk-Aware Memory Clearance for Long-Horizon LLM Agents

MeClear uses cooperative Shapley attribution to clear harmful memories from long-horizon LLM agents, boosting task recovery by 25.5 points over baselines.

The paper introduces MeClear, a task-conditioned memory clearance framework for long-horizon LLM agents that identifies and selectively suppresses memories with negative downstream utility without permanently altering the persistent memory bank. It combines Leave-One-Out screening with sampled cooperative Shapley attribution to distribute utility across interacting evidence, resolving redundant conflict masking that single-removal evaluations miss. Across ten long dialogue memory pools it achieves 85.9% target recall and 82.3% overall task recovery, a 25.5 percentage-point improvement over LOO baselines.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:46 UTC · 7d agoAI research

32

EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?

Researchers introduce EVOHARNESSBENCH, a benchmark showing that evolving agent harnesses (tools, skills, agents) cause forgetting and inconsistent adaptation across 802 tasks.

The paper introduces EVOHARNESSBENCH, a benchmark that places non-stationarity in the externally supplied agent harness rather than in the task stream, evaluating agents across tools, skills, and specialist agents. It comprises 17 multi-stage harness streams built deterministically from verifier-based benchmarks, totaling 802 tasks, 520 tools, 42 skills, and 62 agents. Evaluation covers deployment (retention of previously accessible competence) and self-evolving adaptation settings. Results show harness expansion alone degrades previously solved tasks (harness-induced forgetting), adaptation gains are inconsistent, and retention and adaptation can pull in opposite directions.

Hugging Face daily papers·Sep 2, 20:00 UTC · 13d agoAI research

40

The Attention Triangle in Audio-Video Models

Researchers analyze the 'attention triangle' in audio-video diffusion models, showing bias-driven cross-attention routing causes semantic leakage and proposing inference-time interventions that improve grounding.

A study probes the three cross-attention edges linking text, audio, and video streams in audio-video diffusion models. It finds the audio-video edge is bidirectional and shaped by parameter-encoded biases, so prompts in tension with learned priors can be overridden, producing visually canonical but incorrect outputs. Attention-derived signals are used as diagnostics and to guide inference-time interventions that improve cross-modal semantic grounding while preserving generation quality.

Hugging Face daily papers·Sep 2, 20:00 UTC · 13d agoAI research

24