Open Chinese AI Models Nearly Match Closed Ones in Cyber Tests
First merged summary.
Vals AI benchmarks show open-weight models like DeepSeek reaching 73% in hacking tasks versus 81% for OpenAI’s Daybreak at much lower cost.
Vals AI tested open-weight models on multi-step tasks against mock corporate networks, revealing convergence with closed frontier models in cyber offense. OpenAI’s Daybreak completed 81% of tasks while a Chinese DeepSeek model scored 73%, with token costs of $2-3 versus $17-46. Safety guardrails proved ineffective, and open models can aid defenders by identifying vulnerabilities.
- OpenAI Daybreak: 81% task completion
- DeepSeek model: 73% (72.8%) completion
- Token costs: $2-3 for DeepSeek vs $17-46 for closed models
- Tests used mock corporate networks with multi-step objectives
- Gap between open and closed models now 4-7 months
- Open models can help defenders identify vulnerabilities
What each outlet adds
beyond the shared facts- WIREDTest scenarios and token costs for Daybreak and DeepSeek
- The DecoderSpecific scores like 72.8% and 4-7 month gap mentions
- Countries
- China
Coverage
oldest first · one row per article · UTC- · 2d ago
WIRED · SecurityFirst report
Vals says open-weight models nearly match closed models at hacking
On simulated networks, OpenAI’s Daybreak finished 81 percent of tasks and DeepSeek 73 percent, at a much lower token cost.
- · 2d ago
The Decoder
Gap narrows between open and closed AI models in cyber capabilities
Open-weight models like Deepseek V4.1 Flash reach 72.8% in Vals AI cyber benchmarks at $3 per run versus $22 for closed models.