arXiv introduces LongBanana benchmark for long-context image synthesis
Original title: LongBanana: An Expert-Verified Benchmark for Long-Context Multi-Reference Image Synthesis
LongBanana contains 90 tasks with 4,670 expert-specified evaluation items for agent-driven multi-reference image generation.
The paper presents LongBanana, the first expert-designed and expert-verified benchmark for long-context multi-reference image synthesis. It includes 90 tasks organized by a 3-by-3 taxonomy, each with about 32 reference images, 20 constraints, and 50 evaluation items, totaling 4,670 atomic items. Evaluation uses graph-grounded design converting obligations into checklist items for content fidelity, relational binding, and global constraint satisfaction. LongBanana-Gen harness framework shows RPR workflow yields gains in evidence grounding.
- 90 carefully designed tasks
- 4,670 expert-specified atomic evaluation items
- 15% double-labeled subset with 89.4% agreement
- graph-grounded evaluation for content fidelity, relational binding, and global constraints
- minimal agent harnesses improve over direct generation
From the original article
1 min read at arxiv.orgWe introduce LongBanana, to our knowledge the first expert-designed and expert-verified benchmark for long-context multi-reference image synthesis in the age of agent-driven image generation. The task requires retrieving dispersed visual evidence, rejecting distractors, binding entities and relations, and composing one coherent final image from multiple references.
By Haoxiang Cao · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.