Study evaluates DeepSeek V4.1 Flash-1017 loss-of-control risks
Original title: Evaluating DeepSeek V4.1 Flash-1017 for loss of control and harmful manipulation risks
On 34 benchmarks it improves on earlier V4 models and nears several closed-source baselines, with residual behaviors still flagged.
An independent evaluation assesses DeepSeek V4.1 Flash-1017, described as a new-architecture update of V4.1 Flash with scaled training and strong agentic capabilities. Using the EU General-Purpose AI Code of Practice and the AEF-1 standard, the authors cover evaluation integrity, loss of control, and harmful manipulation. They report substantial safety gains over the V4 generation and results that approach or match GPT-6 Astra, GPT-5.6 Sol, and Gemini 3.8 Flash on several dimensions, while notable behaviors remain.
Unconfirmed: Safety rankings are the authors' benchmark results, not confirmed incidents or vendor admissions.
- The suite has 34 benchmarks, including five in-house suites.
- Comparators include V4.1 Flash, earlier DeepSeek V4 models, and three closed-source systems.
- Authors say safety can shift or regress across minor updates.
- They argue final-turn checks miss loss of control without traces and action logs.
- They warn public benchmarks invite evaluation awareness and data contamination.
- Vendors
- DeepSeek
From the original article
1 min read at huggingface.co
As frontier language models grow increasingly capable and autonomous, independent third-party evaluations are critical for characterizing their systemic safety risks. While closed-source releases are routinely accompanied by comprehensive system cards, open-weight models receive less scrutiny regarding frontier risks. To address this gap, we present an independent safety evaluation of DeepSeek V4.
By Yao Huang, Zhiyuan Zeng, Yu Wang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.