AV-FABLE benchmarks sparse audio-visual manipulations in long videos
Zero-shot audio-visual language models are near chance, while separated readouts of frozen features lead localization.
Stories per day · 30 days
2 RSS
Relatedbenchmark2deepfake1edge-inference1forensics1long-form-video1speaker-extraction1speech-processing1
11 stories, newest first · one card per story
Stories tagged #audio-visual
Falcon cuts causal AV speaker extraction cost on new RealSSA benchmark
Falcon reports 91.9% fewer parameters and 11.1 times lower GPU latency than causal AV-TFGridNet while topping RealSSA-Sim extraction quality.
SyncReward: Human-Aligned Synchronization Reward for Audio-Visual Generation
SyncReward adds an 80,000-rating dataset and reward model for audio-visual synchronization.
OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning
OmniCapBench diagnoses fine-grained audio-visual captioning errors across 786 annotated videos.
OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
OmniSeek trains Omni-LLMs to seek audio-visual evidence across turns using a 170K trajectory corpus.
LEAP: Learned Block-wise Evidence Retrieval for Long Audio-Video Perception
LEAP retrieves short audio-visual windows for long recordings, beating Qwen3-Omni by up to 16.8%.
OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
OmniReasoning-30B-A3B lifts audio-visual joint reasoning by 9.3 points over Qwen3-Omni using new data and self-distillation.
Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models
SECRET, a training-free steering method, cuts source-confused hallucinations in audio-visual language models.
HelixWorld: A Real-time Interactive Audio-Visual World Model
HelixWorld is a real-time audio-visual world model running joint rollouts at 24 FPS.
Omni-Streaming Thinking
Omni-Streaming Thinking fixes premature cross-modal commitment in streaming omni-modal models via pending claims verified against modality-specific evidence, beating baselines by over 10%.
Realtime-Venus: A full-duplex interaction system with asynchronous delegation
Realtime-Venus introduces two 9B full-duplex interaction models (Omni and Audio) that outperform Gemini 3.1 Live and GPT-4o on continuation metrics.