AVC-Reward: Event-Centric Audio-Video Consistency Evaluation
AVC-Reward scores complex audio-video consistency at event level and aligns with human judgments better than prior metrics.
The authors argue existing audio-video metrics fail on multi-event scenes, layered audio, and asymmetric cross-modal relations. AVC-Reward is an event-centric reward model that scores global narrative harmony, event-level temporal synchronization, and fine-grained attribute alignment. AVC-Benchmark is built mainly from positive and negative samples generated by various audio-video models. AVC-Reward reports state-of-the-art agreement with human judgment, localizes failures, and improves a joint generator when used as an online reinforcement-learning signal.
- Metric covers narrative harmony, temporal sync, and attribute alignment.
- AVC-Benchmark uses positive and negative samples from multiple generators.
- AVC-Reward reports state-of-the-art agreement with human judgments.
- Used as an online RL signal, it improved a joint audio-video generator.
- Products
- AVC-Benchmark
- AI models
- AVC-Reward
From the original article
1 min read at arxiv.orgAs joint audio-video generation models become more capable, they can generate audio-visual scenes exhibiting three forms of complexity: multi-event composition, multi-layer audio mixture, and asymmetric cross-modal relations. However, current evaluations largely reuse metrics from earlier simple and unidirectional tasks.
By Yuruo Wu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.