Video-Coder trains code-driven agents for video generation
Original title: Video-Coder: Reinforcing Agentic Coding for Video Generation
Video-Coder achieves 7-point gains on PhyGenBench and 24-point gains on VBench-2.0 via agentic reinforcement learning
Video-Coder is a code-driven agent that constructs 3D scenes for downstream video generators. It uses SFT followed by harness-native RL with dual rewards and an annealing strategy. The method improves performance on video generation benchmarks while the authors open-source data, models, and code.
- Datasets: Video-Coder-SFT-6k and Video-Coder-RL-2k
- Harness integrates code execution and 3D rendering
- Annealing reduces execution reward over training
- Improvements measured on PhyGenBench and VBench-2.0
From the original article
1 min read at huggingface.co
Recent years have witnessed remarkable progress in video generation in terms of visual quality and temporal consistency. However, generating videos with precise spatial structure, coherent motion dynamics, and controllable scene composition remains challenging.
By Chaoyang Wang, Sicheng Gao, Junan Hu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.