Reasoning-Informed Visual Editing
RISEBench++ finds even the best visual editor, GPT-Image-2.5 Sunburst, reaches only 56.6% accuracy.
RISEBench++ evaluates reasoning-informed visual editing across temporal, causal, spatial, logical, counterfactual, and hybrid reasoning, with 12 subcategories and 65 task types. It includes 1,000 human-annotated cases, multi-image inputs, and English and Chinese releases, judged on instruction reasoning, appearance consistency, and visual plausibility. Across 58 systems, the best score is 56.6% for GPT-Image-2.5 Sunburst; the training-free RISE-Agent beats most existing approaches.
- RISEBench++ spans six reasoning types, 12 subcategories, and 65 task types.
- The set has 1,000 human-annotated cases released in English and Chinese.
- Authors evaluated 58 systems, including 34 open-source and 19 closed models.
- Strongest result is GPT-Image-2.5 Sunburst at only 56.6% accuracy.
- Training-free RISE-Agent plans, uses tools, and refines with a verifier.
Full article238 words · extracted from huggingface.co · click to collapse
Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but still face challenges in visual editing, particularly in following complex instructions, preserving appearance consistency, and supporting flexible input formats. To study this gap, we introduce RISEBench, the first benchmark for evaluating Reasoning-Informed viSual Editing (RISE), and extend it to RISEBench++, a more comprehensive and fine-grained benchmark for this emerging task. RISEBench++ extends the taxonomy into a hierarchical scheme spanning six reasoning dimensions: Temporal, Causal, Spatial, Logical, and Counterfactual Reasoning, together with Hybrid Reasoning integrating multiple reasoning types across multi-turn edits. These dimensions are further decomposed into 12 subcategories and 65 fine-grained task types. We expand input formats to include multi-image conditioning and scale the benchmark to 1000 human-annotated test cases, released in English and Chinese. We also improve our evaluation framework, assessing Instruction Reasoning, Appearance Consistency, and Visual Plausibility with human judges and an LMM-as-a-judge approach for more reliable and calibrated judgements. Beyond benchmarking, we introduce RISE-Agent, a training-free agentic framework integrating reasoning-driven planning, tool-augmented execution, and verifier-guided refinement, outperforming most strong existing approaches across diverse RISE tasks. We evaluate 58 visual editing approaches, including 34 open-source models, 19 closed-source models, and 5 agentic methods. The results reveal substantial challenges in reasoning-based visual editing, with even the strongest evaluated approach, GPT-Image-2.5 Sunburst, achieving only 56.6% accuracy. RISEBench++ highlights the limitations of contemporary editing models, provides insights, and indicates future directions for reasoning-aware visual editing.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2610.12343