CWE-bench v2 adds more coverage quality and transparency
CWE-bench v2 release expands coverage quality and transparency.
19 stories, newest first · one card per story
CWE-bench v2 release expands coverage quality and transparency.
On a 526-function PrimeVul test set it reported 75.5% accuracy and 77.2% F1, ahead of DeepSeek V4 Pro and Gemini 2.5 Pro on those metrics.
Across 162 Reddit posts and 180 GitHub issues, 521 weakness instances cluster in access control, exposure, injection, and unsafe execution.
Tuned LLM detectors reach macro F1 0.61 on a realistic test set with a 3% vulnerability rate.
On CWEval, open-weight models gain 15.5% FS@1 over RAG, and Claude Sonnet 5.5 reaches 92.1%.
It matched official vectors on 48.8% of 36,325 usable 2024 CVEs after training only on 2022 and 2023.
ToFu fuzzing found verified semantic bugs in 343 LLM tools across 118 MCP servers.
Cisco issued an October 2026 NX-OS hardening release for internally found flaws not known to be exploited.
Failure-driven prompt refinement uses recurring LLM mistakes to improve vulnerability analysis on DVJA and Juliet.
Across 32 LLMs, newer models are safer in absolute terms, but no family closes the code security gap.
A new metric tracks static-analysis security debt across intermediate states of LLM coding agents, not just final code.
SecProbe adaptively tests coding agents on synthesized vulnerability repairs; frontier models peak at 28.33% success.
Cisco released Nexus Dashboard hardening updates for multiple internally discovered vulnerabilities, grouped by CWE and not known to be exploited.
NIST Bugs Framework evaluation shows it is more structured and automation-friendly than CWE for automated vulnerability classification, with gaps in attribute guidance.
VLoc Bench tests 27 language models at locating vulnerable files in 290 repositories; best system reaches 0.229 File F1 and 38.4% of tasks unsolved.
Study of 126 onboard satellite OSS repositories finds 2,827 security findings, 72% medium severity or higher, dominated by memory safety and code quality weaknesses.

AWS publicly released its Deception Benchmark (14,822 samples) showing leading AI models falsely flag 41-99% of safe code as vulnerable.
BUGSTONE-E2E converts CVE patch history into executable LLM-guided detection rules, yielding 1,033 rules and 644 runtime-verified findings across 14 programs.

CISA's FY2024-2025 vulnerability review urges eliminating recurring vulnerability classes, finding 41.5% of 2025 KEV entries map to persistent 'stubborn weaknesses'.
Today 1.5K readers · 2.4K views · 754 reads · all time 189K views