MEND: RL For Flow Models via Proximal Velocity Matching
MEND post-trains flow models with proximal velocity matching, beating Flow-GRPO and ReFL in far fewer updates.
MEND is a reinforcement-learning method for reward post-training of flow models that uses proximal velocity matching instead of long KL-penalized reweighting or unchecked reward backpropagation. It caps rewards within each prompt group, proposes moves along the reward gradient only below the cap, and accepts a move only when the capped gain exceeds a quadratic displacement cost, then regresses onto the resulting velocity targets. In 100 updates it outperforms Flow-GRPO, which uses about 4,000 updates, on five of six evaluators at equal distance to base-model images, and under an equal budget it surpasses ReFL and DiffusionNFT, reaching PickScore 24.03 versus 23.92 and 23.43.
- Moves are accepted only when capped reward gain exceeds a displacement price.
- Training uses no KL term, frozen reference, or advantage weights.
- PickScore reaches 24.03 versus 23.92 for ReFL and 23.43 for DiffusionNFT.
- The method applies to any flow backbone with a differentiable reward.
From the original article
1 min read at huggingface.co
Reward post-training of flow models either reweights the model's own samples under a KL penalty or a frozen reference, often for thousands of updates, or backpropagates the reward and moves every sample without checking that the move is worth its size. We introduce MEND, a reinforcement learning method built on proximal velocity matching.
By Shreshth Saini, Neil Birkbeck, Yilin Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.