Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
Composing preference and rubric rewards lifts post-trained Flux2dev and Ideogram-4 on the Arena text-to-image leaderboard.
The authors post-train open-domain text-to-image models by composing a Bradley-Terry preference reward, trained on large-scale human preferences, with rubric rewards for prompt faithfulness and safeguards against reward hacking. They report that a naive weighted average optimizes poorly and propose a composition strategy that balances preference and rubric terms. On the Arena text-to-image leaderboard, RL-trained Flux2dev gained 69 Elo over its base model, and post-trained Ideogram-4 reached Elo 1223.5, ahead of every open-source model in a September 4, 2026 snapshot. They also release Arena-T2I-Training, a 1K subset that recovers part of the full-data gain.
- Preference reward uses a Bradley-Terry objective on human data
- Rubric rewards target faithfulness and limit reward hacking
- Flux2dev gained 69 Elo; Ideogram-4 reached 1223.5 Elo
- Arena-T2I-Training releases a 1K subset of training data
- AI models
- Flux2devIdeogram-4
From the original article
1 min read at huggingface.co
Recent text-to-image generation models have achieved remarkable visual quality, but improving them through post-training remains challenging because no single reward signal captures the full range of human preference. In this work, we develop a simple and effective post-training recipe for open-domain text-to-image generation based on the composition of complementary reward signals.
By Yuanhao Ban, I-Hung Hsu, Anastasios Angelopoulos · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.