OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
OmniReasoning-30B-A3B lifts audio-visual joint reasoning by 9.3 points over Qwen3-Omni using new data and self-distillation.
The paper introduces OmniReasoningBench, with 1,150 multiple-choice and open-ended questions where both audio and visual evidence are required, plus the OmniQA engine that builds evidence-grounded pairs and timestamped clue chains. Training sets OmniReasoning-SFT-112K and OmniReasoning-RL-19K support Modality-Factored Self-Distillation for token-level credit assignment. OmniReasoning-30B-A3B scores 50.0% on OmniVideoBench and 42.5% on OmniReasoningBench, improving Qwen3-Omni-30B-A3B-Thinking by 12.8 and 9.3 points, with further gains on Video-MME-v2.
- OmniReasoningBench has 1,150 questions requiring joint audio-visual evidence.
- OmniQA generates SFT-112K and RL-19K sets with timestamped clue chains.
- MFSD assigns token-level credit under modality-specific clue contexts.
- The 30B-A3B model gains 9.3 points on OmniReasoningBench over Qwen3-Omni.
- Vendors
- Alibaba
From the original article
1 min read at huggingface.co
Recent advances have enabled unified omni-modal models in understanding audio, vision, and language. However, existing benchmarks, training data, and learning methods largely treat the modalities independently, leaving the capability of audio-visual joint reasoning poorly evaluated and insufficiently elicited. We address this gap with a benchmark, data engine, and learning method.
By Junming Lin, Yuxuan Wang, Zhenxin Lei · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.