Skip to content
NewA calmer, faster ZeroHour

Interactive Visual Gym evaluates visual assistants in closed loop

Original title: Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds

Hugging Face daily papersby Chenhao Zheng, Zixian Ma, Kai Li ()1 min read1
AI summary · x-ai/grok-4.7Summaries by Vani AI

Gemini 4 Argon finishes 53.4% of 315 scenarios in time; a human study finds 78.5% metric agreement.

The paper presents Interactive Visual Gym, a closed-loop framework that jointly simulates executable task environments and human users for visual interaction models. It includes simulators for cooking, household manipulation, and mobile interfaces, plus a persona-conditioned user harness that can make controlled errors. IVG-Bench has 315 scenarios. Gemini 4 Argon, the best model tested, completes 53.4% of tasks within the time limit and is often too slow to prevent errors. A proof-of-concept GRPO run improves Qwen3.5-4B success and interaction quality in the cooking environment.

Unconfirmed: Benchmark scores and the human-study agreement are reported by the authors and not independently verified here.

Key facts
  • The best model is weak on factual grounding and actionable guidance.
  • A human study finds 78.5% agreement with the interaction-quality metric.
  • Closed-loop GRPO training improves Qwen3.5-4B in the cooking environment.
  • Offline replay benchmarks cannot capture how assistant replies change later user actions.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at huggingface.co

Visual interaction models that support users during ongoing physical tasks require evaluation in closed loop, where assistant responses can change subsequent user behavior and task outcomes. Existing benchmarks largely rely on offline replay and therefore cannot capture these effects.

Hugging Face daily papers

By Chenhao Zheng, Zixian Ma, Kai Li · huggingface.co

Read the full article at Hugging Face daily papers

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.