Three AI models agree on one of 121 potential code flaws
Original title: Three AI Models Agree on Just One of 121 Potential Code Flaws
Echo said 100 findings came from a single model, and the audited projects remain unnamed.
Security firm Echo compared Claude Mythos 5.1, GPT-5.6 Sol and GLM-5.3 on the same two Maven libraries and one npm package. Of 121 distinct potential issues, all three models flagged only one, and 100 came from a single model. Forty-two findings passed automated proof-of-concept testing, but Echo said those were not necessarily distinct bugs and maintainers have not confirmed them. Project names and technical details stay unpublished until notification.
Unconfirmed: Echo called the exercise a controlled comparison, not a comprehensive benchmark, and said automated validation is not maintainer confirmation.
- Claude Mythos logged 25 validation passes, GLM-5.3 nine and GPT-5.6 Sol eight.
- Nine runs produced 446 initial findings, reduced to 152 after cleanup.
- GPT-5.6 Sol finished hunts in about 24 minutes; Claude Mythos took about 198.
- Echo's hunt costs were about $22 for GLM-5.3, $30 for GPT-5.6 Sol and $303 for Claude Mythos.
- One npm package has more than 350 million monthly downloads; models disagreed on one flaw's severity.
- Organizations
- Echo
- AI models
- Claude Mythos 5.1GPT-5.6 SolGLM-5.3
From the original article
2 min read at infosecurity-magazine.com
Echo tested Claude Mythos, GPT-5. 6 Sol and GLM-5. 3 on the same code and found little overlap
By Alessandro Mascellino · infosecurity-magazine.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.