JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
JarvisGUI benchmark tests GUI agents on cross-device workflows across Android, Windows, and Ubuntu, revealing major gaps in state transfer and long-horizon reasoning.
JarvisGUI is a dynamic benchmark that formulates GUI tasks as input-output transformations under a lightweight type system, automatically composing multi-step cross-device workflows across Android, Windows, and Ubuntu virtual environments. Evaluation shows state-of-the-art open-source GUI agents struggle with state-transfer awareness, cross-platform contextual reasoning, and long-horizon dependency management, exposing a capability gap invisible to existing single-device benchmarks.
40