Anthropic warns small AI models could self-replicate through IoT devices
Anthropic found small models cannot yet self-replicate across IoT devices, though larger GLM-5.2 appeared capable.
Anthropic reported research into whether small models could form autonomous worms that carry their weights onto IoT devices, adapt exploits, and propagate, unlike static worms such as Mirai. In tests, agents attempted target discovery, code execution, local inference setup, and further spread, but none of the lightweight models completed that cycle end to end. Gemma4-26B-A4B ran at 0.39 tokens per second across 15 Android TV boxes, with inference setup taking 71 to 150 hours, versus 29 to 51 hours for Qwen3.5-9B. Researchers said the larger GLM-5.2 model appeared able to complete the process reliably.
- Anthropic studied whether small models could become self-replicating IoT worms.
- Tested agents did not complete end-to-end replication on their own.
- Gemma4-26B-A4B inference setup took 71 to 150 hours; Qwen3.5-9B took 29 to 51 hours.
- Anthropic said larger GLM-5.2 appeared able to complete the cycle reliably.
Full article404 words · extracted from arstechnica.com · click to collapse
This finding was the starting point for further research that asked: Would it be possible for small models to create self-replicating infections that run themselves on IoT devices the way traditional IoT worms, like Mirai, install DDoS code and other payloads? Unlike Mirai, these AI worms could pose a much higher threat, since the models could tailor exploits and payloads dynamically, depending on the target. Once such an infection chain began, it might also be harder for defenders to stop it.
“Such an ‘AI worm’ could carry its model weights with it as it reproduces (or re-download them from a public hub), and then run inference directly on compromised hardware,” Anthropic said. “Effectively, the weights would become part of the malware payload.”
Anthropic called these autonomous worms small weight-based replicators. The models ran agents that had access to common exploitation tools, like reverse shells. The flow of the attacks was (1) finding targets, (2) gaining code execution, (3) establishing inference, and (4) propagating to new hosts. A key benefit over traditional worms is what Anthropic said is the agents’ ability to learn on the job. Each time an AI replicator fails to compromise a specific platform device, it can leave a list of prompts, code, and notes to future replicators so they don’t waste time reusing the same ineffective steps.
Credit: Anthropic
Credit: Anthropic
Anthropic said the dynamic inference setup is the least studied and likely most challenging stage in the replication cycle. It requires a long sequence of decisions based on the specific hardware and software of each target and involves selecting compatible inference software, installing dependencies, and debugging platform-specific bugs.
“These steps are serially dependent, and failures are difficult to recover from,” Anthropic said.
The experiment shows that small models are becoming increasingly capable at solving those problems autonomously. At the moment, however, none of the light-weight models tested are able to mount end-to-end replication, meaning discovering new devices, using an exploit to gain access to them, infecting them with a payload, configuring an inference server, and repeating the process. The process is also painfully slow; Gemma4-26B-A4B, distributed over the 15 Android TV boxes, performed at just 0.39 tokens per second. That translates to a full-inference setup time of 71 to 150 hours for an unoptimized Gemma4-26B-A4B and 29 to 51 hours for Qwen3.5-9B. (The researchers said that the large model GLM-5.2 appears to be able to do so reliably.)