YANAMI Dataset and Benchmark Released for Multimodal Stateful Interaction
Original title: YANAMI: A Large-Scale Multimodal Dataset and Benchmark for Multiturn Stateful Interaction
YANAMI introduces the first large-scale multimodal dialogue dataset with 400K multi-turn interactions across four scenarios.
YANAMI addresses the lack of supervision for multimodal stateful interaction in unified models. The dataset YANAMI-400K contains 400K multi-turn dialogues with over 2M images and 2.6M turns spanning multi-image fusion, image editing, world modeling, and visual reasoning. YANAMI-Bench evaluates 400 multi-turn dialogues on dialogue state understanding and visual realization. Fine-tuning BAGEL on the dataset improves image editing, generation, and multimodal stateful interaction by 10.3%, 2.0%, and 13.23% respectively.
- YANAMI-400K has more than 2M images and 2.6M dialogue turns.
- Four scenarios: Multi-Image Fusion, Image Editing, World Modeling, and Visual Reasoning.
- YANAMI-Bench evaluates 400 multi-turn dialogues.
- Fine-tuning improves image editing by 10.3%.
- AI models
- BAGEL
From the original article
1 min read at huggingface.co
Unified Multimodal Models (UMMs) integrate visual understanding, generation, and editing within a single unified framework. However, existing unified multimodal datasets and benchmarks remain dominated by single-turn tasks. Even in recent multi-turn datasets, many turns can be solved independently without tracking information across turns.
By Shengyu Wang, Tengyue Nong, Junming Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.