Elon Musk’s X and xAI companies are under formal investigation by the UK’s data protection watchdog after the Grok AI tool produced indecent deepfakes without people’s consent.
The Information Commissioner’s Office is investigating whether the social media platform and its parent broke GDPR, the data protection law.
It said the creation and circulation of the images on social media raised serious concerns under the UK’s data regime, such as whether “appropriate safeguards were built into Grok’s design and deployment”.
The move came after French prosecutors raided the Paris headquarters of X as part of an investigation into alleged offences including the spreading of child abuse images and sexually explicit deepfakes.
X became the subject of heavy public criticism in December and January when the platform’s account for the Grok AI tool was used to mass-produce partially nudified images of girls and women. The standalone Grok app was also used to generate sexualised deepfakes.
X and xAI announced measures to counter the abuses, but several regulatory and legal investigations have followed. The two companies have been approached for comment.
The executive director of regulatory risk and innovation at the ICO, William Malcolm, said: “The reports about Grok raise deeply troubling questions about how people’s personal data has been used to generate intimate or sexualised images without their knowledge or consent, and whether the necessary safeguards were put in place to prevent this.
“Losing control of personal data in this way can cause immediate and significant harm. This is particularly the case where children are involved.”
GDPR requires that people’s data, including their image, is managed fairly, lawfully and transparently – and that they are informed about how their data is used.
Breaches of GDPR can result in a fine of up to £17.5m or 4% of global turnover. X’s revenues are not public, but according to estimates from the market research company eMarketer it was expected to make $2.3bn (£1.7bn) in advertising turnover last year, which would equate to a fine of about $90m.
Iain Wilson, the managing partner at the law firm Brett Wilson, said: “The ICO’s investigation raises serious questions about the nature of AI-generated imagery and how it is sourced. If photographs of living individuals have indeed been used to generate non-consensual sexual imagery, then it is difficult to imagine a more egregious breach of data protection law.
“This is particularly so if the subjects are identifiable or children.”
Grok AI generated about 3m sexualised images in less than two weeks, including 23,000 that appear to depict children, according to researchers.
X is owned by xAI, the developer of Grok, although both have been subsumed into Musk’s SpaceX rocket business under a merger announced on Monday.
In a separate statement, the UK’s communications regulator, Ofcom, said it was not investigating xAI, which provides the standalone Grok app.
Ofcom also said its investigation into X was still gathering evidence and that the inquiry could take months.
The company has taken steps to address the issue and must be given a “full opportunity to make representations”, it said.
On why it was not investigating xAI, the statement said not all chatbot activities were covered by the Online Safety Act, the legislation that covers sites such as X. If a chatbot interacts with one individual and no other users, for example, it is not within the scope of the act.
Pornography providers, however, are covered by the act, leaving Ofcom with a potential route to widen its investigation. It said it was considering an investigation into whether xAI complied with rules requiring the age-gating of pornographic content.
Meanwhile a cross-party group of MPs led by Labour’s Anneliese Dodds has written to the technology secretary urging the government to introduce AI legislation to prevent a repeat of the Grok scandal. The legislation will require AI developers to thoroughly assess the risks posed by their products before they are released.
“The scandal would not have happened in the first place if proper testing and risk assessment had been undertaken. This episode shows existing safeguards are not sufficient,” said Dodds.
