Deepfake ban: What AI providers in the EU could face from the end of 2026

AI applications for the abusive creation of sexualized deepfakes are to be banned in the EU in the future. Representatives of the member states and the European Parliament agreed on a corresponding adjustment of the AI ​​law, as the Cypriot EU Council Presidency announced. At the same time, other AI rules are to be simplified in order to exploit the economic potential of artificial intelligence in Europe.

Before the changes can come into force, the agreement must be confirmed by the plenary session of the EU Parliament and the Council of Member States’ Governments. As a rule, this is considered a formality. If the reform goes through, the ban would be enforced from December 2, 2026 by the EU’s AI Office, which was created two years ago.

In so-called deepfakes, a woman’s face is inserted into another video – such as a porn sequence – or her voice is artificially imitated so that it appears as if she is doing or saying things that never actually happened. The development of AI has made creating deceptively real content much easier.

The new ban should also explicitly target the creation of content that depicts child sexual abuse. The FDP MEP Svenja Hahn welcomed the agreement: “AI must not be a tool for sexual violence against children,” she said after the negotiations that lasted late into the night.

Grok scandals and German debate about digital violence

At the EU level, the issue came into focus at the end of last year thanks to the AI ​​chatbot Grok: Until the US company behind the software, run by tech billionaire Elon Musk, restricted this function, people repeatedly ordered the AI ​​to undress women in images they selected. On New Year’s Eve, the chatbot itself apologized for creating an image of two teenage girls “in sexualized outfits.”

In Germany, the debate about sexualized digital violence took on new urgency at the end of March when Collien Fernandes went public with allegations against her ex-husband Christian Ulmen. These allegations are not about deepfakes. Fernandes accuses him of creating fake profiles in her name and using them to distribute pornographic content. The presumption of innocence applies to elms.

Since the allegations became known, there has been a major nationwide discussion about digital and sexual violence against women – and widespread media coverage of it. Thousands of people took to the streets to demonstrate and demand more protection for victims. In connection with this debate, sexualized deepfakes and deepfake pornography were repeatedly discussed. Material like this has been circulating online for years.

The EU presented a directive on the subject as early as 2024

The planned tightening of the law is not the first EU rules to combat digital violence. Existing regulations already stipulate that the creation and distribution of manipulated depictions of sexual acts without the consent of those affected should be punishable in all member states.

Although the corresponding EU directive has been in force since May 2024, Germany has not yet transposed it into national law. The Federal Republic still has until the summer of next year to do this. Most recently, Justice Minister Stefanie Hubig (SPD) announced corresponding tightening of criminal law and more rights for those injured.

The new ban in the AI ​​regulation at EU level that is now emerging would change the perspective from punishing the crime to banning the tool, i.e. the AI ​​application itself. Participants in the negotiations emphasize that the ban is not intended to excessively restrict the creation or manipulation of images.

The new law should precisely define what is sexualized content and what is not. Non-consensual bikini pictures like those created by Grok and distributed on X could therefore remain permitted.

Mandatory watermarks for AI content postponed

The EU Commission originally proposed changes to the AI ​​law in order to relieve the burden on the economy and in particular the AI ​​industry. However, companies have recently repeatedly asked for more time to implement the necessary adjustments to meet the stricter requirements. According to the agreement, providers of chatbots and other services should be given this time.

The companies behind ChatGPT, Claude and Co. now only have to comply with parts of the law that the European AI Office was originally supposed to enforce from August onwards from December 2026. By then at the latest, providers should clearly mark AI content as such – so they would have to mark generated images and videos with watermarks. The AI ​​office is only scheduled to enforce other rules from December 2027.

Industry fears double regulation

There have recently been calls from industry to reduce double regulations in order not to undermine Europe’s competitiveness. Chancellor Friedrich Merz (CDU) also recently spoke out in favor of simplifying the European rules in this area. To ensure that the mechanical engineering industry does not have to adhere to several EU regulations at the same time when using AI, exceptions should be created.

For CDU MEP Axel Voss, these changes do not go far enough: “We need a framework that enables innovation and guarantees protection – not a patchwork of sectoral AI special rules,” commented the parliamentarian.