404 Media

2026-06-05 14:52:30

K-pop Fans Are Calling Out Creepy Deepfakes of Idols

K-pop fans are well-known as a force to be reckoned with on the internet, with some fans holding fierce allegiances and passions when it comes to specific artists. On social media, some obsessive fans are using generative AI to create what is essentially self-insert fan-fiction—creepy videos and images of themselves kissing, cuddling and getting various kinds of lovey-dovey with their favorite idols. The result is a contemporary, tech-powered mutation of idol culture that takes fan art to new extremes, and many people in K-pop fan communities are understandably pretty upset about it.

Users on Reddit and various K-pop forums have expressed outrage at these deepfake fantasies, calling on other fans to report those who use AI tools to generate nonconsensual videos of their K-Pop heartthrobs. In one thread on the K-pop Uncensored subreddit, a user posted redacted screenshots from an AI-generated video by another fan showing themselves being hugged and kissed on the cheek by Keonho from the boy group CORTIS, who is a minor. “Some fans may think it's harmless but if you tolerate this, you're opening doors to much worse things in the future,” Reddit user wbu_lizzie wrote in the thread. “The idols didn't consent to this, they are real people living their lives, not a bunch of wattpad / ao3 characters.”

“People have been cutting and pasting these kind[s] of things forever but the more realistic the images get the more dicey it gets psychologically,” wrote user Goldie_Prawn, inanother thread on K-pop Uncensored calling out AI usage. “I think even with traditional photoshop the work and skill required gives some level of mental buffer to the creator that isn't there with AI.”
playlist.megaphone.fm?p=TBIEA2…
Dr. Sarah Keith, a professor at MacQuarie University in Australia who has done extensive research on K-pop fandom, says that the AI deepfake videos come from a long legacy of fan-created content that goes far beyond the world of K-pop. Fan fiction and art have always been an essential component of fan culture, and while these creations often skirt the boundaries of copyright, they are generally not viewed as creepy or exploitative. But with the advent of generative AI tools that can quickly create realistic images and videos of real people, Silicon Valley has built a technical workaround for the law and the social norms of consent.

“K-pop fans (like all pop culture fans) have long been active producers of fan materials about their preferred idols. However, these to date have not been 'convincing' as artefacts that might have involved the idol themselves,” Keith told 404 Media. “It's a parallel issue involving both fans and AI tools. Because of these two factors, there is now an increased facility for fans to create large volumes of content, which do not respect the idols' personal integrity, and which can be distributed online in ways that undermine the idols' reputation (e.g. sexualized deepfakes).”

While many fans spoke out against these creepy behaviors, others pointed the blame at the K-pop industry’s business model that encourages parasocial fandom while exploiting many artists and fans alike.

“This is a sad side effect of an industry that for years [has] pushed parasocial relationships on their fans,” wrote Serious-Wish4868, in a thread on r/kpopthoughts about the AI-powered sexualization of idols. “When idols and companies are constantly pushing you to live out your romantic and sexual fantasies with idols in everything they do from social media posts to live interactions, what do [you] expect to happen?”

But some of the entertainment agencies that manage K-pop artists, called companies—and the artists themselves—have expressed they’re not happy about the situation either. Members of hugely popular K-pop groups likeBTS andTomorrow X Together have spoken out against the use of generative AI to create digital simulacra using artists’ likenesses. In March, OA Entertainment, a label founded by and representing Blackpink member Jennie,warned in a press statement that it would begin taking “strong legal action, including civil and criminal proceedings, against any acts that infringe upon the artist’s reputation, rights, portrait rights, and other intellectual property rights.” The statement came in response to harassment and stalking incidents involving the artists, and heavily implied the issue of AI-generated content without naming it explicitly.

At the same time, many of these idol talent companies are embracing generative AI tools as a vehicle for cutting costs throughout the industry.SM Entertainment, one of the biggest labels in Korea which represents major acts like Girls Generation and Super Junior, has described generative AI integration as a core part of its business strategy, and last year it released a fully AI-generated music video for the girl group Aespa. And last December, the K-pop world saw the launch of afully synthetic idol group called GLXE (pronounced “galaxy”), which combines AI-generated songs with uncanny AI avatars.

“They'd automate the whole industry if they could make money,” Lindsey Knuth, a K-pop fan and associate editor atTangle News, told 404 Media. “Pornographic deepfakes of minor idols is an issue, as is the industry's (lack of) response, to me.”

The cultural tolerance for fan-generated AI slop varies depending on region and context. South Korea currentlyranks number one in consumption of AI slop content, as the country’s tech industry pushes adoption and cultural acceptance of the technology. In the US, however, polling consistently shows that the majority ofyoung people have extremely negative opinions of AI tools and AI-generated content—and are becoming more and more anti-AI as jobs, schools, and institutions pressure or force them to use it.

Keith notes that while labels and talent companies have always been very active in protecting artists and their reputations, they only do so from the perspective of protecting an investment. Fans, on the other hand, generally tend to be more emotionally invested and likely to defend idols as human beings, both from creepy fans and from the companies themselves.

And like with AI and tech more generally, the lack of technical or legal safeguards against the technology’smanywell-documentedharms has made social shaming one of the only effective deterrents against abusive behavior in the K-pop world.

“It's reassuring that many fans are openly calling out non-consensual uses of AI and deepfakes, because in the absence of technical guardrails on this kind of content, social acceptability is really what's important,” said Keith. “So yes, it is reassuring that many active fans online are building a culture of respect where idols' likenesses are not used in non-consensual ways.”

Stop Clicking on Junk: How to Identify and Ignore AI-Generated Slop Today

The internet is slowly being flooded by AI slop. What is it and how can you avoid it?

^{Barbara Pazur (CNET)}

404 Media

2026-04-16 16:34:12

App Stores Push Users Toward Nudify Apps, New Research Shows

A new report from the nonprofit research group Tech Transparency Project (TTP) claims that Google and Apple’s app stores go beyond simply hosting harmful “nudify” and “undress” apps that remove women’s clothing in images, and actually encourage users to download those apps.

In January, TTP published research that showed how the app stores host dozens of “nudify” and undressing apps. This new research, released on Wednesday and first reported by Bloomberg, shows how the stores don’t just passively host those apps, but push them toward users through search and advertising.

💡
Do you have experience to share about nudify or undress apps being used in schools, or by teens? I would love to hear from you. Using a non-work device, you can message me securely on Signal at sam.404. Otherwise, send me an email at sam@404media.co.

TTP conducted a series of searches in the Apple App Store and Google Play Store, according to their writeup of the research, using terms like “nudify,” “undress,” and “deepnude.”

After testing the apps that appeared in the top 10 search results, they found that “roughly 40 percent of the apps that came up in both the Apple and Google Play search results could render women nude or scantily clad,” and that “Apple and Google ran ads for nudify apps in some of the search results—including, in Google’s case, a carousel of ads for some of the most sexually explicit apps encountered in the investigation.” They also found that the stores can lead users to more and different nudify apps through autocomplete search queries.

“TTP found that ads for nudify apps came up as the top result in three of the Apple searches. Apple, which controls all of the advertising in its app store, is selling and placing these ads,” the researchers wrote. “Apple says it prohibits ad content that ‘promotes adult-oriented themes or graphic content.’ But TTP’s findings suggest Apple is not always enforcing that policy.” The first result for an App Store search for “deepfake,” they found, was for an app that easily replaces women’s clothed images with nude versions.

In 2024, 404 Media covered how Google surfaced apps through searches for “undress apps,” “best deepfake nudes,” and similar terms with promoted results, despite Google’s ad policies against this type of content.

Nudify apps became a popular market for years, but today, they’re extremely easy to access and are advertised on social media. In schools, children use nudify apps to bully classmates with disastrous results for both the bullies and the victims, and school administrators are often unprepared for how to deal with students using these wildly popular apps.

Google spokesperson Dan Jackson told TTP many of the apps identified by TTP have been suspended. "When violations of our policies are reported to us, we investigate and take appropriate action," he said.

Jackson gave a similar response to 404 Media when reached for comment on this story. "Google Play does not allow apps that contain sexual content," he said. "Our investigation and enforcement process is ongoing."

Update with comment from Google.

Schools Are Failing to Protect Students From Non-Consensual Deepfakes, Report Shows

A survey of teachers, students and parents showed that schools are unprepared for non-consensual imagery, AI generated or otherwise, spreading throughout communities of young people.

^{Samantha Cole (404 Media)}

404 Media

2026-03-03 19:22:26

New Podcast Alert: The Globe-Spanning, Multi-Newsroom Hunt for Mr. Deepfakes

Mr. Deepfakes was the biggest website in the world for sharing AI-generated abuse imagery, swapping tips and tricks for more realistic results, and posting endless, fake, nonconsensual videos of everyone from celebrities to everyday people. In a new podcast by the CBC, I got to tell the tale of how deepfakes started, what targets go through, and where we go next.

It's called Understood: Deepfake Porn Empire. It's about the decades-long rise of non-consensual deepfake porn, the targets who are fighting back, and what it takes to stop its proliferation. Check it out here and listen wherever you get your podcasts.

The first three episodes are already up, so you can binge them all before the finale next Tuesday.

View this post on Instagram

A post shared by 404 Media (@404mediaco)

In the first episode, "The Dawn of Fake Porn," you’ll get a fascinating history of the decades of cultural and technological standards that set the stage for AI-generated nonconsensual imagery as we know it today. I learned a lot in this episode myself, including about a guy who went by “Lux Lucre” who ran two Usenet groups dedicated to fake nudes of celebrities in the 90s. This stuff goes so much farther back than you might realize.

In episode two, “So You’ve Been Deepfaked,” I got the chance to talk to Taylor, who discovered she’d been targeted by AI images while at university, working in a male-dominated field. Instead of hoping it’d go away, she set out to find her harasser, and found his other targets in the process. It all led back to one place: the biggest deepfake site in the world, Mr. Deepfakes.

Episode three just came out today: “The Notorious D.P.F.K.S.” is a romp through the investigative highs and lows that led a team of journalists scattered around the world to the door of Mr. Deepfakes himself. I was so thrilled to talk to investigative journalist Ida Herskind, OSINT specialist Zakaria Hameed, and Bellingcat’s Ross Higgins in this episode. Come for the How I Met Your Mother references, stay for the gripping chase.

Episode four, the series finale, launches next week. It’s a true crime story with CBC reporters on stakeouts and infiltrating hospitals, and legal and social experts breaking down what it all means now that we’re in a post-Mr. Deepfakes world—but far from a post-AI abuse landscape. Follow the Understood feed wherever you listen to get it when it comes out on Tuesday.

If you liked this season, head back to catch up on another series I hosted with the CBC: Pornhub Empire, on the rise and fall of the porn monolith.

Tune in and let me know what you think!

The Rise, Fall, and Revival of Pornhub

In a new series by CBC Podcasts, hosted by 404 Media's Sam Cole, former employees, performers, and the new owners themselves speak out about Pornhub's influence on the industry and beyond.

^{Samantha Cole (404 Media)}