DiffusionART: Efficient, Stable and Scalable Reinforcement of Diffusion Models
DiffusionART stably reinforces diffusion models without EMA tuning, matching HunyuanVideo-13B rewards with about 12 times fewer GPU hours.
DiffusionART is an online reinforcement method for diffusion models that aggregates each prompt's rewards into a shared reward-improving direction and adds a trust-region-masked update for drift-adaptive restoration without EMA tuning. The authors argue NFT-style objectives contain a forward–reverse consistency force that joint positive–negative training only partly cancels, and that EMA stabilizes training by restoration rather than off-policy rollouts. It requires no differentiable reward model, works with fewer samples per prompt, and supports pairwise rewards. On HunyuanVideo-13B with equally weighted visual and motion rewards, it reaches late-stage mean rewards of DiffusionNFT and Flow-GRPO-Fast with 12.02 and 11.72 times fewer GPU hours, and 28.8% and 53.0% higher mean training rewards.
- DiffusionART aggregates per-prompt rewards into one reward-improving direction.
- It needs no differentiable reward model and avoids EMA tuning.
- On HunyuanVideo-13B it uses 12.02x and 11.72x fewer GPU hours than DiffusionNFT and Flow-GRPO-Fast.
- Mean training rewards are 28.8% and 53.0% higher than those baselines.
- AI models
- HunyuanVideo-13B
From the original article
1 min read at huggingface.co
Online reinforcement learning (RL) fine-tunes diffusion models with black-box rewards, but reverse-process methods such as Flow-GRPO are limited in efficiency by poor credit assignment, while forward-process methods such as DiffusionNFT are efficient yet unstable without carefully tuned exponential-moving-average (EMA) off-policy training.
By Zhenglin Wan, Jiayu Chen, Zhengfeiyang Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.