MoI-Bench scores mobile image generation and editing gaps
Original title: MoI-Bench: A Benchmark for Image Generation and Editing on Mobile Devices
The suite has 64 subtasks and 1,150 samples, with MoI-Eval scoring instruction following, quality and detail preservation.
Researchers introduce MoI-Bench to compare on-device and server-side image models against mobile user needs and device limits. Sixteen operations and eight visual target categories define tasks such as removing facial makeup. MoI-Eval adds task-specific criteria and a Quality-Deployability map. Results say compact models need broader task coverage and better identity preservation and text rendering more than further efficiency gains.
- The benchmark contains 64 subtasks and 1,150 samples.
- Operations and targets form two orthogonal task axes.
- Authors flag text replacement as a specific capability gap.
- On-device models can generate or edit efficiently but lack coverage.
From the original article
1 min read at arxiv.orgDeveloping on-device image models requires considering practical user needs, task types and device resource constraints. However, existing benchmarks provide limited guidance on identifying capability gaps between on-device models and server-side models. We introduce MoI-Bench, a benchmark grounded in mobile user needs, comprising 64 subtasks and 1150 samples.
By Kailai Feng · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.