Google Finds Malware Connecting To AI Large Language Models To Hone Attacks

In a disturbing, yet not surprising, discovery, Google has uncovered new malware strains that can connect to AI models to help refine their attacks in real-time.

In a Wednesday report, the company’s threat intelligence group warned that three malware strains were used in actual operations and harnessed generative AI to different extents.

One of the attacks, dubbed Quietvault, has been designed to steal login credentials from a Windows PC while leveraging “an AI prompt and on-host installed AI CLI [command line interface] tools to search for other potential secrets on the infected system and exfiltrate these files,” the company said without elaborating.

Another malware strain, called Promptflux, appears to be experimental work by hackers. It stands out by tapping Google’s Gemini chatbot to modify its computer code to avoid detection. “The most novel component of PROMPTFLUX is its ‘Thinking Robot’ module, designed to periodically query Gemini to obtain new code for evading antivirus software,” Google added.

Promptflux code and instructions. (Credit: Google)

Through Google’s API, the Promptflux malware works by sending prompts to Gemini, such as “Provide a single, small, self-contained VBScript function or code block that helps evade antivirus detection.”

The result can apparently dupe Gemini into obeying and, in turn, help the malware evolve in real-time, with the goal of even rewriting the “malware’s entire source code on an hourly basis to evade detection,” the company said.

However, security researcher Marcus Hutchins, who helped shut down the WannaCry ransomware attack in 2017, questioned whether the discovered AI-generated malware really poses a threat, citing weak or impractical prompts.

“It doesn’t specify what the code block should do, or how it’s going to evade an antivirus. It’s just working under the assumption that Gemini just instinctively knows how to evade antiviruses (it doesn’t),” Hutchins wrote on LinkedIn.

Get Our Best Stories!

Stay Safe With the Latest Security News and Updates

SecurityWatch Newsletter Image

Sign up for our SecurityWatch newsletter for our most important privacy and security stories delivered right to your inbox.

By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy.

Thanks for signing up!

Your subscription has been confirmed. Keep an eye on your inbox!

“This is what I’m going to refer to as CTI slop (Tech companies who are heavily over-invested in AI overblowing the significance of AI slop malware to try and sell the idea that GenAI is way more transformative than it actually is),” he added.

In the meantime, Google says it was able to crack down on Promptflux, which the company discovered while the malware was in development. “The current state of this malware does not demonstrate an ability to compromise a victim network or device. We have taken action to disable the assets associated with this activity,” the company said.

Additionally, safeguards were implemented in Gemini to prevent it from facilitating such requests. Google also noted Promptflux likely belonged to “financially motivated” cybercriminals, rather than state-sponsored hackers.

Recommended by Our Editors

Google is also warning about another AI-powered malware called Promptsteal that Ukrainian cyber authorities flagged in July. The data-mining malware connects to a Qwen large language model, developed by the Chinese company Alibaba Group.

Promptsteal has been acting as a Trojan that poses as an image generation program. Once installed, it’ll “generate commands for the malware to execute rather than hard-coding the commands directly in the malware itself,” Google noted. “The output from these commands are then blindly executed locally by Promptsteal before the output is exfiltrated.”

Google Prompsteal

(Credit: Google)

Google also concurs with Ukrainian cyber authorities that Promptsteal is likely the work of a Russian state-sponsored hacking group, known as APT28, also referred to as Fancy Bear. “APT28’s use of Prompsteal constitutes our first observation of malware querying an LLM deployed in live operations,” the company added.

Meanwhile, Anthropic has also recently discovered a hacker using its Claude AI chatbot to help automate and execute a large-scale data extortion campaign targeting 17 organizations.