TaoMate-HD streams high-resolution audio-video with progressive denoising
Authors report a 1.6x speedup versus direct high-resolution generation and an 11.18-point drop in character error rate.
Stories per day · 30 days
2 RSS
Relatedevaluation2multimodal2reinforcement-learning2reward-model2autoregressive1benchmark1denoising1diffusion1digital-humans1editing1multimodal-generation1streaming1taomate-hd1training1
10 stories, newest first · one card per story
Stories tagged #audio-video
UniAVC trains one model for joint audio-video creation and editing
Each branch first learns from unimodal data; scarce paired data is reserved for cross-modal coupling.
AVC-Reward: Event-Centric Audio-Video Consistency Evaluation
AVC-Reward scores complex audio-video consistency at event level and aligns with human judgments better than prior metrics.
AVC-Reward evaluates audio-video consistency at the event level
Authors say the reward model aligns best with human judgments and can steer a joint generator via online reinforcement learning.
SepGen: Multi-Stem Audio-Video Separation and Generation in a Single Model
SepGen jointly generates video, a mixed soundtrack, and per-source audio stems, and can also separate mixes from captions.
Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training
Prism uses dynamic sparse attention to train joint 2K video-audio models 2.5× faster than full attention.
Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation
Kandinsky 6.0 Video releases open 3B and 29B models for synchronized text- and image-to-audio-video.
Salt++: Context-Aligned Post-Training for Few-Step Streaming Multimodal Generation
Salt++ post-trains few-step streaming audio-video models, beating OmniForcing on quality and LTX-2 on most metrics.
All modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generation
RecCAR regularizes cross-attention so companion modalities better constrain joint video generation.
The Attention Triangle in Audio-Video Models
Researchers analyze the 'attention triangle' in audio-video diffusion models, showing bias-driven cross-attention routing causes semantic leakage and proposing inference-time interventions that improve grounding.