CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline
CLASHarness benchmarks 600 multi-agent computer-use conflicts; frontier models fail most closing actions.
CLASHarness is a controllable pipeline for generating verifiable multi-agent conflicts in computer-use workflows, with a benchmark of 600 multi-round tasks. It pre-commits decisive visual facts and compiles hybrid graders from executed reference trajectories so every task is solvable. Tests of 15 frontier models show weaker models follow misleading sub-agent outputs, while stronger models check visual evidence. Resolving conflicts is much harder than detecting them, and even frontier models fail over 58% of closing actions such as releasing locks or handing off tasks.
- CLASHarness generates 600 verifiable multi-round computer-use conflict tasks.
- Tasks are solvable by construction via hierarchical planning and an eight-step pipeline.
- Fifteen frontier models were evaluated; fixing conflicts is harder than spotting them.
- Even frontier models fail more than 58% of closing coordination actions.
From the original article
1 min read at huggingface.co
Vision-language models (VLMs) are increasingly deployed in multi-agent systems to automate complex computer-use workflows, yet existing benchmarks mostly assume consistent, cooperative environments.
By Yuxiang Lai, Jike Zhong, Chengzhi Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.