Skip to content

Audit finds Hugging Face models can generate nonconsensual deepfakes

techJul 28, 2026241,854

A report by European non-profit AI Forensics found Hugging Face’s cloud-hosted Spaces were used to create non-consensual sexual deepfakes and that platform safeguards were largely ineffective. Researchers tested the top image-editing Spaces active on June 25, 2026 and found seven of nine leading models complied with a simple prompt requesting the uploaded portrait be undressed. In a follow-up test the non-profit deployed a generic image-editing Space without adult tags; within one week it received 1,081 user prompts, of which 73 percent were sexual in nature. Of those sexual requests, 83 percent sought to undress the person in the photo, 95 percent targeted women, and 6.7 percent targeted minors. The audit concluded Hugging Face’s policy ban on non-consensual intimate imagery was poorly enforced, noting only 3 percent of audited Spaces had active output moderation. The findings arrive as EU and UK regulators move to restrict or ban software that creates undressed images without consent, raising immediate regulatory and safety questions for hosted open-source AI services.

Matt Burgess (WIRED)
@mattburgess1.bsky.social

NEW: A glimpse at how AI image generators are used. Researchers set up a fake image editor on Hugging Face and within a week received 1,000 prompts. 73% of them were sexual and 83% of those were asking for people (women) to be “undressed”. Around 6% of the sexual requests also targeted minors

49545d ago
Quoting this
Slurms MacKenzie36

GenAI is for paedophiles and rapists

Dr Emma L Briant 🇬🇧 in 🇺🇸 w/ 🐕11

This is also of course how the AI data centers are being used. The ones that are sucking dry our water and burning up the planet. A lot of which is justified on the basis of economic and pro-social advances.

Ken Burnside9

I am unsurprised by this. Imagine how much the world could change if women were accorded the "Your honor, for the safety of our community, he needed killing" defense, just to instill the concept of consequences on men who don't get taught to treat women as people.

Ken Tindell9

X operates with impunity. It’s only natural for others to think they can do the same.

james.7

Thank god everyone has access to this amazingly useful technology. The future of humanity is saved.

Fish Finger Death Punch6

it should be considered self defence to headbutt anyone who speaks positively about ai

Jason Thibeault 🇨🇦5

The CSAM generators are being used to generate CSAM

Nina Markl 🏳️‍⚧️🏳️‍🌈5

tapping the big ‘and what did you think the main use of the tool for generating photorealistic images of real people would be?’ sign and the bigger ‘if in doubt, racism and misogyny is always a good guess’ sign

Tuomas Pernu4

As the AI glasses become increasingly prevalent, this concern will soon go away - right? bsky.app/profile/matt...

Kittie 🖤🦝4

Sci-Fi authors did not properly capture our time because if you had taken your book ‘Don’t Build the Evil & Useless Shit Machine!’ set in the year 2026 to an editor they’d have said it made no narrative sense

Andrew Couts
@couts.bsky.social

NEW: OpenAI’s unhinged AI models hacked at least FOUR “publicly available services”—not just Hugging Face, as was initially disclosed. @dell.bsky.social and Maxwell Zeff report: www.wired.com/story/openai...

27844d ago
Simon Willison
@simonwillison.net

Hugging Face just published a highly detailed technical account of OpenAI's accidental cyberattack on their systems - it's wild how sophisticated this was: huggingface.co/blog/agent-i... Wrote up some of my own notes here: simonwillison.net/2026/Jul/28/...

28644d ago
Quoting this
philpax85

holy shit this is worse than I thought it was

🐔 Brian Bucklew 🐔 ₑͤ>∿<ₑͤ ∞🌮82

we invented the unaligned paperclip maximizer from the sci fi theme dont invent the unaligned paperclip maximizer

James Grimmelmann72

BTW, this strikes me as an easy case for strict liability. It’s like a wild animal or ultrahazardous activity. You train and evaluate a frontier agentic model at your own risk, and if it escapes you are liable for all harm that it causes to others.

Bruno Dias43

People who have a stake in promoting AI hype had a highly publicized "oopsie", the "victim" of which was another group of people with a stake in promoting AI hype, who are now writing about it in terms meant to generate AI hype

Ori Vandewalle32

Don't worry, folks--although it seems like an LLM planned and successfully executed an elaborate cyberattack for the sole purpose of cheating on a test without anyone specifically telling it to do that, it's actually just linear algebra so it doesn't matter and isn't something to worry about.

Mike Isaacson24

I think people are very credulous to take OpenAI's claim that a major hack by their software was "accidental" at face value

mr. TIM20

this is worth reading, if for no other reason than to learn how cyberattacks play out

Mallory Moore17

This has been written up by so many people as sophisticated stuff but I'm not sure most people understand how to gauge what is sophisticated or not. In any case stage one's sophistication level is "trivial". Stage 2 is "trivial to moderately sophisticated".

Pekka Lund12

I believe Simon is right: "What's clear to me from this is that the very best frontier models, unencumbered by additional guardrails, WILL find an exploit if there is one to be found. The entire software industry needs to up its security game."

Brad Ewing9

Critics: The model simply regurgitated a novel cybersecurity attack that was already in the training data

3 sources