iPrefBench tests when mobile agents should infer or ask
Seven frontier agents hit up to 91.6% action completion but only 47.7% full task success.
Stories per day · 30 days
Relatedbenchmark2mllm2agent-security1computer-use1cross-modal1ios1jailbreak1mobileinject1personalization1prompt-injection1qwen1
7 stories, newest first · one card per story
Stories tagged #mobile-agents
MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injection
MobileInject benchmarks MLLM mobile agents against multimodal injection across 108 risk instances, finding broad vulnerability.
MobileJailBench benchmarks jailbreaks against MLLM mobile agents
The 520-task suite spans 14 risk categories and reports 100% ASR on Qwen3.5-122B after dynamic evolution.
OpenMobile-2 open mobile agent trained on 110-plus apps
Authors report AndroidWorld 79.9, MobileWorld 50.4, and SPA-Bench rising from 31.9 to 59.6.
Jev-Mobile: Jev as an Executor for Mobile GUI Agents
Jev-Mobile splits VLM planning from fast GUI execution, cutting AndroidWorld API cost 73% at similar success.
Qwen-Planner-Agent: A Closed-Loop AI-for-AI Framework for Real-World Mobile Planner Agents
Qwen-Planner-Agent uses a closed-loop AI-for-AI pipeline and leads MobilePA-Bench for mobile planning.
When Agents See Differently: Exposing UI Desynchronization Threats in Mobile Agents
Researchers expose 'human-agent UI desynchronization' attacks where repackaged APKs invisibly mislead mobile AI agents into attacker-chosen actions.