The recent summary proceedings brought by the Offlimits Foundation against X and its AI chatbot Grok have created a stir, if only by the far-reaching consequences of their outcome. On 12 March 2026 in the Court of Amsterdam, the interest group that fights online transgressive behaviour and notably child sexual abuse took on the tech giant because of the nudification content – also of minors – that can be created with Grok in a trice. Offlimits won; the functionalities of Musk’s AI chatbot for creating this content category were prohibited for Dutch users.
Heart of the matter
In collaboration with Stichting Fonds Slachtofferhulp, Offlimits examined so-called ‘nudify’ websites in 2025.[1] These are websites where the imagery of (real) persons can be used to fully or partially ‘undress’ them. On the websites examined, this could be done without checking whether the real existing persons whose data were used had expressly consented to this. Offlimits argued that generating such imagery is contrary to Dutch and EU laws, including the General Data Protection Regulation (“GDPR”).[2]
Grok is a generative AI chatbot developed by X.AI. Grok is freely available for consumers via an app, web page, and the social media platform X in which Grok is incorporated. The chatbot can perform different orders, including generating imagery. Offlimits has established that Grok also enables users to generate and then reuse ‘nudification imagery’: photographs and videos with moving images.
The heart of the matter is that Grok makes it possible to undress a (real) person fully or partially without obtaining the express consent of the person concerned for this. Grok has set the age limit for persons with whom this functionality can be used at 10 years old, which means that users may even generate imagery with Grok that qualifies as child pornography. In addition, Grok contains the ‘Spicy mode’, which is specifically intended for sexualizing generated content. The investigations of Offlimits have also revealed that these images can be uploaded easily via X, which makes their distribution child’s play.
Offlimits chooses summary proceedings via class action route
A remarkable procedural aspect is that Offlimits chose to bring its claims via the class action regime of Section 3:305a DCC and the WAMCA [Act on Collective Damages in Class Actions] regime based on it. The writ of summons shows that Offlimits expressly used a non-commercial class action aimed at the immediate termination of a continuing unlawful situation. Offlimits stressed that the main intention of its claims was to prevent future victims and that proceedings on the merits would not be sufficiently effective to achieve this. According to Offlimits, regular WAMCA proceedings on the merits would take several years, given experiences in previous class actions, while research[3] by the Center for Countering Digital Hate has shown that the alleged infringements kept occurring daily in high numbers. That was the precise reason why Offlimits felt that urgent relief was necessary.
Moreover, Offlimits believes that choosing the collective route was a functional necessity since the claims are mainly intended for future victims. Without a class action, Offlimits argued that “the victims would be left empty-handed” and no effective ban against the functionality as such could be obtained.
In the summary proceedings, Offlimits only claimed the immediate prohibition of Grok’s unlawful and criminal nudify functionality, at least as far as it concerns real persons residing in the Netherlands. More specifically, the claims concerned a ban on generating and/or distributing sexual imagery that uses the functionality to undress people fully or partially without their express consent. This ban was only a disciplinary measure to prevent future infringements by ceasing the infringing acts of Grok and X immediately. The purpose of the action was not to obtain damages or the worldwide removal of unlawful imagery.
Precisely because no monetary damages were claimed, Offlimits could rely on the “light regime” of Section 3:305a (6) DCC. The Court has largely followed this line of reasoning. In its judgment, the Court held that the action was in the public interest and had a non-commercial purpose: preventing and fighting online sexually transgressive behaviour and online sexual child abuse, fully in line with the objective in the articles of association of Offlimits. Since the claims did not include compensation and only sought to protect general and future interests, the Court considered the light regime of Section 3:305a (6) DCC to be justified.
Offlimits did not only base its claims on the GDPR but also on the fundamental right to protection of privacy, unlawful act, the Dutch Copyright Act and the Digital Services Act (DSA).
Does the AI Regulation offer solace?
The AI Regulation[4], which prohibits several AI systems in view of their intended purpose, has been partially effective (in the Netherlands) since February 2025. It is remarkable that the EU legislator gave no direct leads for a ban on an AI system like Grok. It is true that Article 5 (prohibited AI practices) contains an exhaustive list of AI practices that are prohibited by definition in the European Union, such as manipulation, abuse of vulnerabilities, social scoring, criminal predictions, biometric categorisation, and untargeted face scraping. However, Grok’s nudification functionality cannot so easily be ranged under those.
The digital “undressing” of persons via an AI system without their consent and the generating of child pornography imagery are undesirable without doubt, but the AI Regulation offers no legal basis to deal with this. Grok’s nudity functionalities are not included either in the statutory ‘high-risk’ AI systems in the AI Regulation. Moreover, the AI Regulation contains no general moral or human-rights ban on harmful applications of generative AI. The legal basis for the claims of Offlimits therefore had to be found in other laws.
It should be mentioned here that things have now started to move at EU level, partly due to the Grok reality. On 6 May 2026, the European Parliament and the Council reached a political agreement concerning the ‘Digital Omnibus on AI’, which provides for an extension of Article 5 of the AI Regulation.[5] This extension introduces an express ban on AI systems capable of generating or manipulating non-consensual realistic sexually explicit imagery of identifiable persons. The explanation to this proposal shows that such applications are considered a severe impairment of fundamental rights, including human dignity, physical integrity and private life. It emphasizes that generating non-consensual intimate imagery must be regarded as a form of sexual abuse that may occur on a large scale and cause considerable psychological harm.[6]
This development confirms that the current AI Regulation offered no express lead for a ban on this type of functionality and shows that the European legislator now appreciates the problems behind this.
Prohibited AI under the GDPR
As mentioned, Offlimits partly based its claims on a violation of the GDPR. The Court held that generating non-consensual undressing footage is contrary to the GDPR. Besides, the Court held that generating child pornographic materials is contrary to a rule of unwritten law relating to proper social conduct within the meaning of Section 6:162 (2) DCC.[7] The Court emphasized that X.AI, as an ‘all internet intermediary’, has de facto control over Grok’s functionalities as an image generator and is therefore the designated party to prevent the generation and distribution of unlawful content.[8]
The Court also considered that it is obvious that personal data of identifiable persons are processed while generating undressing footage of real persons. These data are faces and bodily characteristics with the use and manipulation of further context. The decisive factor is that the data subject is recognizable and that the processing occurs without any form of consent, contrary to Article 6 GDPR that requires a legal ground for every processing. Data subjects did not give such consent in the context of Grok. There is also no legitimate interest of Grok that outweighs the infringement of the privacy of the data subjects, nor any other ground that would justify the processing. Moreover, the imagery qualifies as ‘special personal data’ within the meaning of Article 9 GDPR. Data on a person's sexual life or sexual integrity are subject to a strict ban on processing. This ban can only be broken as an exception, for example if the data subject has expressly consented. Such consent was fully absent here. This results in an unlawful processing of personal data via Grok by both the user and the provider of the data.
Under the GDPR, both X and X.AI can be designated as controllers for the imagery generated via Grok, including that of the persons who fell victim to the undressing functionality. Offlimits substantiated that this controllership exists by referring to the Russmedia judgment.[9] Albeit without a reasoned explanation on this point, the Court confirmed this division of roles under the GDPR in the judgment.[10] According to the Russmedia judgment, an online marketplace cannot elude liability and responsibility under the GDPR on the ground that it does not determine the substance of the content on its platform.[11] In this case, the ECJ established – in summary – that an online marketplace where sensitive personal data are being processed must verify personally, as the controller, whether the data subject has explicitly consented to this processing. It is imaginable that the Court applied this outcome by analogy to X.AI (and X) as an ‘internet intermediary’.
It is also worth mentioning that Article 5 of the GDPR requires the controller(s) to take appropriate technical and organizational measures for the protection of the personal data processed (via Grok), including against unauthorised or unlawful processing. The Court considers this contrary to the systematic offering of a functionality that is known to lead to large-scale breaches of privacy. The Court held that X and X.AI failed to specify sufficiently that the measures they did take are really and sufficiently effective.
AI Regulation is not the only ground for prohibited AI systems
The legal action against the providers of Grok demonstrates that the AI Regulation containing the ‘prohibited AI systems’ – despite its projected extension with nudification functionalities – need not be the end when it comes to AI-related abuses. As this case shows, an AI system or one or more of its functionalities can be restricted or prohibited by relying on other legal grounds, such as the GDPR for the unlawful processing of personal data.
[1] https://offlimits.nl/assets/offlimits_nl/rapport-nudify-websites.pdf.
[2] Regulation (EU) 2016/679.
[3] https://counterhate.com/research/grok-floods-x-with-sexualized-images/.
[4] Regulation (EU) 2024/1689.
[5] Proposal for a Regulation of the European Parliament and of the Council amending Regulations (EU) 2024/1689 and (EU) 2018/1139 as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI).
[6] Proposal for a Regulation of the European Parliament and of the Council amending Regulations (EU) 2024/1689 and (EU) 2018/1139 as regards the simplification of the implementation of harmonised rules on artificial intelligence (Digital Omnibus on AI), recital 6a.
[7] Court of Amsterdam, 26 March 2026, ECLI:NL:RBAMS:2026:3106, ground 4.20.
https://uitspraken.rechtspraak.nl/details?id=ECLI:NL:RBAMS:2026:3106.
[8] Court of Amsterdam, 26 March 2026, ECLI:NL:RBAMS:2026:3106, ground 4.21.
[9] European Court of Justice (Grand Chamber) 2 December 2025, C-492/23, ECLI:EU:C:2025:935, (X v. Russmedia).
[10] Court of Amsterdam, 26 March 2026, ECLI:NL:RBAMS:2026:3106, ground 4.5.
[11] European Court of Justice (Grand Chamber) 2 December 2025, C-492/23, ECLI:EU:C:2025:935, (X v. Russmedia) https://infocuria.curia.europa.eu/tabs/document/C/2023/C-0492-23-00000000RP-01-P-01/ARRET/307102-NL-1-html
