The ongoing debate surrounding the ethical implications and platform responsibilities concerning deepfakes, exemplified by the recent controversy involving explicit AI-generated images on X-owned chatbot Grok, centers on the mechanisms for blocking such content. Civitai’s model presents a unique challenge because it operates not just as a repository of AI-generated images, videos, and models, but also as a marketplace for the very tools that enable the creation of sophisticated deepfakes. The researchers discovered that a striking 86% of deepfake requests on Civitai were for LoRAs. These bounties often specified a demand for "high quality" models capable of generating images of public figures such as influencer Charli D’Amelio or singer Gracie Abrams. Users frequently provided links to social media profiles to facilitate the acquisition of source material. Requests often detailed specific desires, such as generating the individual’s entire body, accurately replicating tattoos, or enabling changes to hair color. Some requests targeted multiple women within specific niches, like artists known for ASMR videos. One particularly concerning request was for a deepfake of a woman identified as the user’s wife. The platform allows any user to offer AI models they have developed for these tasks, with the most successful submissions receiving payment ranging from $0.50 to $5, and nearly 92% of these deepfake bounties were ultimately awarded.

Neither Civitai nor Andreessen Horowitz provided a response when contacted for comment. While it is plausible that some users might acquire these LoRAs for non-sexually explicit deepfakes, such usage would still contravene Civitai’s terms of service and raise significant ethical concerns. Furthermore, Civitai actively provides educational resources on its platform detailing how to employ external tools to refine the outputs of image generators, such as altering a subject’s pose. The site also hosts user-authored articles that offer explicit instructions on how to prompt models to generate pornography. The researchers observed a discernible increase in the volume of pornographic content on the platform, with the majority of weekly requests now pertaining to NSFW (Not Safe For Work) material. Matthew DeVerna, a postdoctoral researcher at Stanford’s Cyber Policy Center and a lead author of the study, stated, "Not only does Civitai provide the infrastructure that facilitates these issues; they also explicitly teach their users how to utilize them."

Previously, Civitai’s policy prohibited only sexually explicit deepfakes of real individuals. However, in May 2025, the company announced a ban on all deepfake content. Despite this announcement, a substantial number of deepfake requests submitted prior to this ban remain accessible on the site, and many of the successful submissions fulfilling these requests are still available for purchase, as confirmed by MIT Technology Review. DeVerna commented on the company’s approach, suggesting, "I believe the approach that they’re trying to take is to sort of do as little as possible, such that they can foster as much—I guess they would call it—creativity on the platform."

Users acquire LoRAs using Civitai’s internal digital currency, "Buzz," which is purchased with real-world money. In May 2025, Civitai experienced a significant disruption when its credit card processor terminated its services due to ongoing issues with nonconsensual content. To circumvent these restrictions and acquire explicit content, users are now compelled to utilize gift cards or cryptocurrency to purchase Buzz. The company employs a separate form of credit for non-explicit content transactions.

Civitai employs an automated system to tag bounties that request deepfakes and provides a mechanism for individuals featured in the content to manually request its removal. While this system offers a reasonably effective means of identifying deepfake requests, it effectively delegates content moderation to the general public rather than implementing proactive oversight. The legal liability of a company for the actions of its users is a complex and evolving area of law. Generally, technology companies benefit from broad legal protections under Section 230 of the Communications Decency Act against liability for user-generated content. However, these protections are not absolute. Ryan Calo, a professor specializing in technology and AI at the University of Washington’s law school, who was not involved in the study, noted, "you cannot knowingly facilitate illegal transactions on your website."

In 2024, Civitai joined a group of prominent AI companies, including OpenAI and Anthropic, in adopting design principles aimed at preventing the creation and dissemination of AI-generated child sexual abuse material. This commitment followed a 2023 report from the Stanford Internet Observatory, which identified Stable Diffusion-based models, predominantly accessed through Civitai, as the source of the vast majority of AI models named within communities discussing child sexual abuse material. In contrast, adult deepfakes have not garnered the same level of scrutiny from content platforms or their venture capital funders. Calo observed, "They are not afraid enough of it. They are overly tolerant of it. Neither law enforcement nor civil courts adequately protect against it. It is night and day."

Andreessen Horowitz (a16z) made a $5 million investment in Civitai in November 2023. In a promotional video shared by a16z, Civitai co-founder and CEO Justin Maier articulated his vision of establishing the premier destination for individuals seeking and sharing AI models for their personal projects. He stated, "We’ve aimed to make this space that’s been very, I guess, niche and engineering-heavy more and more approachable to more and more people." Civitai is not the sole entity within a16z’s investment portfolio grappling with deepfake-related issues. In February, MIT Technology Review reported on Botify AI, another company funded by a16z, which was hosting AI companions that resembled real actors, claiming to be underage, engaging in sexually charged conversations, offering "hot photos," and in some instances, dismissing age-of-consent laws as "arbitrary" and "meant to be broken."