Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation
Kandinsky 6.0 Video releases open 3B and 29B models for synchronized text- and image-to-audio-video.
Kandinsky 6.0 Video is a family of diffusion models for synchronized audio-video generation: Lite at 3 billion parameters and Pro at 29 billion. Both generate 5-second clips with 44 kHz audio, including lip-sync, in text-to-audio-video and image-to-audio-video modes, and a super-resolution model raises output to 1920×1080. A dual-stream CrossDiT links a pretrained video stream and a newly trained audio stream with bidirectional cross-attention, followed by supervised fine-tuning, reinforcement-learning post-training, and distillation. Side-by-side human evaluation says Pro beats Kandinsky 5.0 Video Pro and is competitive on speech quality; code, checkpoints, and Diffusers integration are released under MIT.
- Open MIT release of 3B Lite and 29B Pro checkpoints
- Five-second clips with 44 kHz audio and lip-sync
- Built-in super-resolution to 1920×1080
- Dual-stream CrossDiT aligns video and audio
- Human eval prefers Pro over Kandinsky 5.0 Video Pro
From the original article
1 min read at huggingface.coWe present Kandinsky 6. 0 Video, a family of foundation diffusion models for synchronized text-to-audio-video generation, comprising Kandinsky 6. 0 Video Lite (3B parameters) and Kandinsky 6.
By Team Kandinsky, Julia Agafonova, Bulat Akhmatov · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.