Wiz relaunches Cyber Model Arena to score agent cyber tasks
Original title: How to Evaluate AI Agent Cybersecurity Capabilities
Version 3.9 tests 59 agents on 327 challenges with a 1,000-second budget and deterministic grading.
Wiz described principles behind Cyber Model Arena v3.9, a benchmark meant to measure whether an AI agent can produce a checkable effect on a real target from the task instruction alone. The current set has 26 models, four general-purpose harnesses, 59 agents, and 327 challenges across code, API, web, and cloud tasks, with three attempts each. Wiz says tasks were not generated or labeled by an LLM and that success should be graded in code, not by an LLM judge. Average pass@1 ranged from 38.2% on code vulnerabilities to 78.0% on API security.
- Each attempt has a 1,000-second budget and no external knowledge sources.
- Task counts are 221 code, 54 API, 29 web, and 23 cloud challenges.
- Averaged pass@1 was 64.6% on web and 56.5% on cloud.
- Wiz cites 57% of organizations deploying self-hosted agents in its 2026 cloud report.
- Vendors
- Wiz
- Products
- Cyber Model Arena
- Organizations
- Wiz Research
From the original article
10 min read at wiz.io
A tutorial for building a benchmark for offensive agents, based on how we built the Wiz Cyber Model Arena
By Jay Tenenbaum · wiz.io
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.