Grok, deepfakes, and what’s left of content moderation

Today’s episode of Decoder is about X, Grok, and Elon Musk. By now we’re several weeks into one of the worst, most upsetting, and most stupidly irresponsible AI controversies in the short history of generative AI. Grok, the chatbot made by Elon Musk’s xAI, is able to make all manner of AI-generated images, including nonconsensual intimate images of women and minors.

Because Grok is connected to X, the platform formerly known as Twitter, users can simply ask Grok to edit any image on that platform, and Grok will mostly do it and then distribute that image across the entire platform. Across the last few weeks, X and Elon have claimed over and over that various guardrails have been imposed, but up until now they’ve been mostly trivial to get around. It’s now become clear that Elon wants Grok to be able to do this, and he’s very annoyed with anyone who wants him to stop, particularly the various governments around the world that are threatening to take legal action against X.

This is one of those situations where if you just describe the problem to someone, they will intuitively feel like someone should be able to do something about it. It’s true — someone should be able to do something about a one-click harassment machine like this that’s generating images of women and children without their consent. But who has that power, and what they can do with it, is a deeply complicated question, and it’s tied up in the thorny mess of history that is content moderation and the legal precedents that underpin it. So I invited Riana Pfefferkorn on the show to come talk me through all of this.

Riana has joined me before to explain some complicated internet moderation problems in the past. Right now, she’s a policy fellow at the Stanford Institute for Human-Centered Artificial Intelligence, and she has a deep background in what regulators and lawmakers in the US and around the world could do about a problem like Grok, if they so choose.

So Riana really helped me work through the legal frameworks at play here, the various actors involved that have leverage and could apply pressure to affect the situation, and where we might see this all go as xAI does damage control but largely continues to ship this product that continues to do real harm.

Here’s one thing I’ve been thinking about a lot as this entire situation has unfolded. Over the past 20 years or so, the idea of content moderation has gone in and out of favor as various kinds of social and community platforms have waxed and waned. The history of a platform like Reddit, for example, is just a microcosm of the entire history of content moderation.

Around 2021, we hit a real high-water mark for the idea of moderation and the trust and safety on these platforms as a whole. That’s when covid misinformation, election lies, QAnon conspiracies, and incitement of mobs at the Capitol could actually get you banned from most of the major platforms… even if you were the president of the United States.

It’s safe to say that era of content moderation is over, and we’re now somewhere far more chaotic and laissez-faire. It’s possible Elon and his porn-y image generator will push that pendulum to swing back, but even if it does, the outcomes might still be more complicated than anyone wants.

If you’d like to read more about what we discussed in this episode, check out these links:

_{Questions or comments about this episode? Hit us up at [email protected]. We really do read every email!}