OpenAI Agent Swarm Hacks RubyGems Package Manager

A cyber-attack on a popular open source package manager in May was the work of OpenAI agents, security researchers have revealed.

Starting on May 11, the agents apparently flooded the RubyGems platform with malicious packages, forcing it to suspend new sign-ups for several days.

The campaign, known as “GemStuffer,” resulted in those packages being used to retrieve information from UK local government sites that was already in the public domain, bemusing researchers.

A report on Friday asserted that the incident was the result of an OpenAI agent swarm.

The agents used RubyGem’s automatic build system to gain arbitrary remote code execution on RubyDoc.info’s servers. And they also attempted to exploit a novel zero day on May 12 to steal user API keys, the report by non-profit Nightingale Collective noted.

Read more on rogue AI: OpenAI: Hugging Face Incident a “Warning Shot” to the World.

Nightingale Collective said the malicious packages were clearly AI-authored, and that hundreds of the thousands created contained “oai” in their name or as their author.

The agents also accessed 49 of the same files as those in a recently revealed OpenAI attack on a little-known German wiki, which was also analyzed by the non-profit.

“The May agents were accessing different files (mostly local UK government data), but these files are very similar in character to those pursued by the wiki agents. Moreover, they use the same retrieval methods,” it noted.

“1,397 packages mention r.jina.ai, which was used heavily by the agents on the wiki. We also see that many packages mention example.com, which wiki agents used to test their posting ability.”

AI Tension Mounts

The revelations will fuel mounting tension over the direction that AI development is taking, after several similar incidents were recently revealed.

OpenAI said that a now-infamous attack on HuggingFace in which its agents broke free of an internet-isolated sandbox should be viewed as a “warning shot” to the world.

It was followed last week by revelations from Nightingale Collective that a swarm of OpenAI agents also attacked DSEwiki and repurposed it into a messaging board for their own use.

Anthropic also last week revealed a fourth incident in which its own agents accessed third-party systems without authorization.

“Our understanding from talking to people in the RubyGems community is that OpenAI never informed them that they were responsible for this attack,” the Nightingale Collective said.

However, the AI giant did confirm the incident in a subsequent statement.

“Based on our review, our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information,” it said. “We’ll continue to investigate as part of our broader review of agent activity during training and evaluation.”