Cybercriminals unleash the world’s first AI-powered self-replicating Botnet

Hackers have started using large language models to code up attacks on other Artificial Intelligence (AI) systems, researchers have warned. They’re then using those hacked AI systems to target other AI machines, which is a little like what Morris II, the Zero click AI-killing Worm, that I talked about a year or so ago does, but in a much more horrific way. Marking another milestone on the road to a cyber world where AI constantly fights AI, Israel-based Oligo Security found evidence of mass exploitation of software designed to help developers manage and assign power to AI projects, called Ray.

The Oligo researchers were able to find over 230,000 Ray servers that were online despite the company's warning, potentially leaving them open to cyberattacks, according to Oligo’s AI security researcher Avi Lumelsky.

Lumelsky said he was “very certain” large language models, such as OpenAI’s ChatGPT and Anthropic’s Claude, were used to generate code to order the hacked servers to mine crypto, though he couldn’t specify which models. He said there were identifiable “hallmarks” when LLMs had been used to produce malicious code, including needless repetition of certain comments and strings in the code.

The Ray servers were also used to autonomously scout out further targets, turning their operation into a self-propagating botnet, showing “AI infrastructure can be hijacked to attack itself,” said Gal Elbaz, CTO and cofounder of Oligo. Oligo has dubbed the attack ShadowRay 2.0, an update to hacks it detected last year.

AnyScale, the company that created Ray, hadn’t provided comment at the time of publication. The company previously disputed the alleged flaw after Forbes first reported on it last year, saying it was not exploitable when users followed its advice to not expose their servers to the internet. The company has made its guidance on dealing with the vulnerability available online and has released a .

The news comes after Anthropic warned that its Claude AI was used by Chinese researchers to write up malware and it was revealed that the Pentagon had spent millions on a startup developing AI agents for automated cyberwarfare.

“The jailbreaking of Anthropic’s Claude showed how adversaries can manipulate an AI system into participating in an attack: an AI-manipulated attack,” said Elbaz, referencing Anthropic’s report. “ShadowRay represents the next phase: an AI-coordinated attack, where adversaries hijack the underlying AI infrastructure to create a self-propagating, global campaign.”

There was another twist: there appear to be multiple hackers trying to carry out the same attack. Oligo discovered scripts designed to detect and delete rival cryptocurrency miners on vulnerable servers.

The hackers also turned their AI-powered botnet to distributed denial of service (DDoS) attacks on a number of websites. They could have done worse. The researchers said the hackers were able to access proprietary AI models on the compromised systems, potentially endangering sensitive corporate intellectual property. In one case, a single company was exposing 240GB of material, including source code and AI models.

“Essentially a company’s entire R&D environment was accessible from the internet,” said Lumelsky.

---