China Asked ChatGPT For Help Crafting Online Harassment Campaigns

Pro tip: If you’re developing a covert influence operation in China, don’t dump all your research into ChatGPT. In a new report, OpenAI says the Chinese government used its chatbot as part of coordinated plans to harass and intimidate political dissidents living in foreign countries, inadvertently giving the American company a view into its efforts to silence critics.

The firm’s researchers identified an account they believe was based in Mainland China that used the AI tool to edit reports about what it called “cyber special operations,” feeding it detailed information and asking for advice.

For example, the Chinese user was looking to discredit Sanae Takaichi, Japan’s first female prime minister, after she criticized human rights abuses in Inner Mongolia. The plan basically consisted of ways to smear the politician via fake online posts, as well as efforts to amplify issues that could hurt her politically. OpenAI says ChatGPT refused to help the Chinese user edit the plan, but elements of it later showed up online.

The scheme also involved reaching out to targets directly. In one example, the Chinese operative reportedly told ChatGPT that a Chinese official had forged documents seemingly from a US court, which they planned to use to dupe a social media platform into taking down anti-Communist Party of China (CCP) content. In another case, Chinese officials impersonated US immigration officials, telling US-based dissidents that their statements had broken the law.

Some of these efforts worked; the Chinese ChatGPT user reported that targeted accounts lost followers or closed up shop altogether. Other efforts had little impact. “The ChatGPT user recorded that their unit had made over 50,000 posts across over 200 Western platforms. Of those, under 150 posts received over 300 shares or comments.”

Impersonation was another common strategy among the Chinese operatives, with five separate fake Bluesky accounts created for a man known as Hui Bo, a well-known Chinese dissident living in California. Fake accounts were also created for Teacher Li, a key figure in the 2022 anti-lockdown protests, who is now living in Italy.

Recommended by Our Editors

In one campaign, the operatives created a fake obituary and fake pictures of gravestones stating that political dissident Jie Lijian had died and posted this content across social media. In another example, AI was used to generate fake “evidence” to support takedown claims against pro-Taiwan X accounts.

OpenAI says it appears the Chinese operation experimented with other AIs, including those developed in the country, like DeepSeek. It also says some of the activity is consistent with the covert influence operation known as “Spamouflage,” a major Chinese disinformation operation identified by Meta in 2023.