Rolling the cyber dice with open-source and open-weight AI models
Open-weight AI models can hide backdoors and poisoning that scanners miss, shifting breach liability onto security leaders.
A CSO Online column argues open-weight models ship parameters without training data or code, so traditional scans cannot find embedded backdoors or poisoning, unlike models meeting the Open Source Initiative’s open-source definition. JFrog has documented malicious pickle-serialized models on Hugging Face that execute code on load, which the author treats as a known packaging problem. Latent behavioral backdoors remain research-stage, including Anthropic’s Sleeper Agents, PoisonGPT, and Winter Soldier, where poisoning under 0.005 percent of tokens—64 documents—implanted a hidden prompt-response pair. The piece says a frontier-model failure sits with OpenAI or Anthropic, while a Hugging Face open-weight failure lands on the CSO, and notes Meta’s 30-billion-parameter Muse Glimmer under Apache 2.0, plus planned Muse Spark 1.2 weights, are open-weight rather than open-source.
- Open-weight releases omit training data and code, unlike OSI open-source AI.
- JFrog found malicious pickle models on Hugging Face that run code on load.
- Behavioral backdoors are shown in research, not documented production breaches.
- Winter Soldier poisoned 64 documents, under 0.005 percent of pre-training tokens.
- Meta Muse Glimmer is a 30-billion-parameter Apache 2.0 open-weight release.
- Countries
- United StatesChina
- AI models
- Muse GlimmerMuse Spark 1.2
From the original article
7 min read at csoonline.com
With typical cybersecurity exposure, I can conduct pen testing with deterministic tools. I am able to predict how a piece of software is going to respond. I even stand a decent chance of finding vulnerabilities before they can be exploited against me.
csoonline.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.