AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation
AV-GRPO applies modality-anchored decoupled diffusion RL to joint audio-video generation, beating LTX-2.3 on quality, alignment, and synchronization.
AV-GRPO is a modality-anchored online diffusion RL framework for joint audio-video generation, pairing three modules (modality-anchored rollouts, trajectory-locked frozen-tower optimization, adaptive objectives) with the 5DAV difficulty-controllable dataset. The approach decouples coupled multimodal preference learning into unimodal subproblems for better credit assignment and synchronization. On JavisBench and VABench, AV-GRPO outperforms LTX-2.3 in generation quality, semantic alignment, and cross-modal synchronization under both LoRA and full fine-tuning. Code and data are released on GitHub.
- Decouples multimodal preference learning into unimodal subproblems for reward attribution
- Frozen-tower optimization cuts compute cost of training two modality towers
- 5DAV dataset varies difficulty across five dimensions; beats LTX-2.3 on two benchmarks
Full article184 words · extracted from huggingface.co · click to collapse
Recent years have witnessed major progress in joint audio-video generation. Existing models still suffer from limited per-modality fidelity, insufficient text-modality alignment and weak cross-modal synchronization. While reinforcement-learning post-training offers a promising remedy, directly adapting it to joint audio-video generation is challenging. Heterogeneous multimodal rewards entangle learning signals and complicate credit assignment. Joint optimization of two modality towers is computationally expensive given their divergent dynamics. Moreover, synchronization evaluation difficulty depends on paired samples, preventing fair reward comparisons. We propose AV-GRPO, a modality-anchored online diffusion RL framework, and 5DAV, a decoupled, difficulty-controllable training dataset. AV-GRPO includes three key modules: (1) modality-anchored rollouts to disentangle learning signals and stabilize difficulty; (2) trajectory-locked frozen-tower optimization to reduce cost and reassign credit; (3) adaptive objectives and perturbation strengths tailored to modality-specific dynamics. This converts coupled multimodal preference learning into unimodal subproblems for precise reward attribution and better synchronization. Our 5DAV dataset decouples samples across five dimensions for systematic training. Experiments on JavisBench and VABench demonstrate AV-GRPO outperforms LTX-2.3 in generation quality, semantic alignment and cross-modal synchronization under LoRA and full fine-tuning. Ablations confirm our designs. Code and data: https://github.com/zhiyuxu03/AV-GRPO
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.29816