Internalizing Agent Experience into Diffusion Model Weights via On-Policy Context Distillation
D-OPCD distills a text-to-image agent harness into diffusion weights, lifting scores from 60.52 to 65.09.
Diffusion On-Policy Context Distillation treats agent-improved prompts as privileged context and distills that harness knowledge into a text-to-image diffusion model's weights. Using an agent with an Auto Skill Evolver, the method raises average direct-generation score from 60.52 to 65.09 across four benchmarks when the model is conditioned only on the original query. After saturated skills are removed from the harness, a second ASE round on the updated generator improves a further 1.83 points over a skill-free harness.
- D-OPCD distills agent-improved prompts into diffusion model weights.
- The generator should keep some harness benefit from the original query alone.
- Auto Skill Evolver supplies the text-to-image agent used for distillation.
- Average direct-generation score rises from 60.52 to 65.09 on four benchmarks.
- A second ASE round adds 1.83 points over a skill-free harness.
From the original article
1 min read at huggingface.co
Wrapping an image generation model in an agentic harness can effectively boost Text-to-Image task performance: the harness can leverage memory, skills, workflow orchestration, result verification, and iterative refinement to continually construct and revise prompts, thereby eliciting better images. These gains, however, remain external to the diffusion model and are realized only while the full harness runs.
By Wenxuan Wang, Zekai Liu, Weinan Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.