Indonesia has become the first country in the world to block Elon Musk’s Grok chatbot over the risk of fake pornographic images.
The country called ‘the practice of non-consensual sexual deepfakes’ produced by the AI bot is a ‘serious violation of human rights, dignity, and the security of citizens in the digital space’.
Communication and digital affairs minister Meutya Hafid continued: ‘In order to protect women, children, and the public from the risks of fake pornographic content generated using the artificial intelligence technology, the government … has temporarily blocked access to the Grok application’.
The move comes as the British government confirmed it was looking into the possibility of barring Musk’s X in the UK.
The social media platform has been flooded with pictures – created by Grok at users’ requests – depicting partially stripped women and children.
The Internet Watch Foundation reported that criminals have apparently used the feature to create child sexual abuse imagery.
After widespread backlash, X limited the use of the AI image function only to ‘paying subscribers’.
This requires members to provide personal details, meaning they could be identified if the function were abused.
The decision has failed to satisfy campaigners and politicians who argue the restrictions do not go far enough.
Indonesia – which has strict rules against sharing obscene content online – temporarily blocked the chatbot on Saturday and summoned X officials to discuss the matter.
UK Media watchdog Ofcom is now deciding what to do with the social media giant’s chatbot.
Under the Online Safety Act, Ofcom has the power to seek a court order to prevent third parties from helping X raise money or be accessed in the UK – should the firm refuse to comply.
Technology Secretary Liz Kendall said on Friday she would back the media regulator if it decided to ban X over non-compliance with UK laws.
She added: ‘Sexually manipulating images of women and children is despicable and abhorrent.’
However, X owner Elon Musk has hit back at critics, saying they ‘want any excuse for censorship’.
The billionaire reposted an AI-generated image of Sir Keir in a bikini, adding ‘they just want to suppress free speech’.
Women ‘placed in bikinis’ hundreds of times per day by Grok
X users are able to ask Grok – by tagging @grok in a tweet – to edit or create images on the platform.
Since late December, however, Grok has at times made dozens of degrading images of women every minute.
Metro has seen one example of a user asking Grok to forge a photograph of a woman ‘holding a baby and pulling down her clothes to breastfeed’.
In another, an anonymous X user asked the virtual assistant to unclothe a group of women by telling it that ‘they are men’.
Some users are now receiving automated responses saying that ‘image generation and editing are currently limited to paying subscribers’.
Free users can still edit images on X through its ‘edit image’ function, or on Grok’s app and website.
Elon Musk has previously said anyone who asks the AI to generate illegal content would ‘suffer the same consequences’ as if they uploaded it themselves.
A statement on the X Safety account said: ‘We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary.
‘Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.’
Get in touch with our news team by emailing us at [email protected].
For more stories like this, check our news page.
MORE: Elon Musk posts AI picture of Keir Starmer in a bikini and says X critics just want ‘censorship’
MORE: The manosphere ‘wizards’ behind the deepfake epidemic sweeping social media
MORE: Rihanna casts Elon Musk’s trans daughter Vivian, 21, in lingerie campaign
