404 Media

2026-02-11 15:16:39

'The Most Dejected I’ve Ever Felt:' Harassers Made Nude AI Images of Her, Then Started an OnlyFans

In the first week of January, Kylie Brewer started getting strange messages.

“Someone has a only fans page set up in your name with this same profile,” one direct message from a stranger on TikTok said. “Do you have 2 accounts or is someone pretending to be you,” another said. And from a friend: “Hey girl I hate to tell you this, but I think there’s some picture of you going around. Maybe AI or deep fake but they don’t look real. Uncanny valley kind of but either way I’m sorry.”

It was the first week of January, during the frenzy of people using xAI’s chatbot and image generator Grok to create images of women and children partially or fully nude in sexually explicit scenarios. Between the last week of 2025 and the first week of 2026, Grok generated about three million sexualized images, including 23,000 that appear to depict children, according to researchers at the Center for Countering Digital Hate. The UK’s Ofcom and several attorneys general have since launched or demanded investigations into X and Grok. Earlier this month, police raided X’s offices in France as part of the government’s investigation into child sexual abuse material on the platform.

Messages from strangers and acquaintances are often the first way targets of abuse imagery learn that images of them are spreading online. Not only is the material disturbing itself — everyone, it seems, has already seen it. Someone was making sexually explicit images of Brewer, and then, according to her followers who sent her screenshots and links to the account, were uploading them to an OnlyFans and charging a subscription fee for them.

“It was the most dejected that I've ever felt,” Brewer told me in a phone call. “I was like, let's say I tracked this person down. Someone else could just go into X and use Grok and do the exact same thing with different pictures, right?”

@kylie.brewer
Please help me raise awareness and warn other women. We NEED to regulate AI… it’s getting too dangerous #leftist #humanrights #lgbtq #ai #saawareness
♬ original sound - Kylie Brewer💝

Brewer is a content creator whose work focuses on feminism, history, and education about those topics. She’s no stranger to online harassment. Being an outspoken woman about these and other issues through a leftist lens means she’s faced the brunt of large-scale harassment campaigns primarily from the “manosphere,” including “red pilled” incels and right-wing influencers with podcasts for years. But when people messaged her in early January about finding an OnlyFans page in her name, featuring her likeness, it felt like an escalation.

One of the AI generated images was based on a photo of her in a swimsuit from her Instagram, she said. Someone used AI to remove her clothing in the original photo. “My eyes look weird, and my hands are covering my face so it kind of looks like my face got distorted, and they very clearly tried to give me larger breasts, where it does not look like anything realistic at all,” Brewer said. Another image showed her in a seductive pose, kneeling or crawling, but wasn’t based on anything she’s ever posted online. Unlike the “nudify” one that relied on Grok, it seemed to be a new image made with a prompt or a combination of images.

Many of the people messaging her about the fake OnlyFans account were men trying to get access to it. By the time she clicked a link one of them sent of the account, it was already gone. OnlyFans prohibits deepfakes and impersonation accounts. The platform did not respond to a request for comment. But OnlyFans isn’t the only platform where this can happen: Non-consensual deepfake makers use platforms like Patreon to monetize abusive imagery of real people.

“I think that people assume, because the pictures aren't real, that it's not as damaging,” Brewer told me. “But if anything, this was worse because it just fills you with such a sense of lack of control and fear that they could do this to anyone. Children, women, literally anyone, someone could take a picture of you at the store, going grocery shopping, and ask AI or whatever to do this.”

A lack of control is something many targets of synthetic abuse imagery say they feel — and it can be especially intense for people who’ve experienced sexual abuse in real life. In 2023, after becoming the target of deepfake abuse imagery, popular Twitch streamer QTCinderella told me seeing sexual deepfakes of herself resurfaced past trauma. “You feel so violated…I was sexually assaulted as a child, and it was the same feeling,” she said at the time. “Like, where you feel guilty, you feel dirty, you feel like, ‘what just happened?’ And it’s bizarre that it makes that resurface. I genuinely didn’t realize it would.”

Other targets of deepfake harassment also feel like this could happen anytime, anywhere, whether you’re at the grocery store or posting photos of your body online. For some, it makes it harder to get jobs or have a social life; the fear that anyone could be your harasser is constant. “It's made me incredibly wary of men, which I know isn't fair, but [my harasser] could literally be anyone,” Joanne Chew, another woman who dealt with severe deepfake harassment for months, told me last year. “And there are a lot of men out there who don't see the issue. They wonder why we aren't flattered for the attention.”

‘I Want to Make You Immortal:’ How One Woman Confronted Her Deepfakes Harasser
“After discovering this content, I’m not going to lie… there are times it made me not want to be around any more either,” she said. “I literally felt buried.”
404 MediaSamantha Cole

Brewer’s income is dependent on being visible online as a content creator. Logging off isn’t an option. And even for people who aren’t dependent on TikTok or Instagram for their income, removing oneself from online life is a painful and isolating tradeoff that they shouldn’t have to make to avoid being harassed. Often, minimizing one’s presence and accomplishments doesn’t even stop the harassment.

Since AI-generated face-swapping algorithms became accessible at the consumer level in late 2017, the technology has only gotten better, more realistic, and its effects on targets harder to combat. It was always used for this purpose: to shame and humiliate women online. Over the years, various laws have attempted to protect victims or hold platforms accountable for non-consensual deepfakes, but most of them have either fallen short or present new risks of censorship and marginalize legal, consensual sexual speech and content online. The TAKE IT DOWN Act, championed by Ted Cruz and Melania Trump, passed into law in April 2025 as the first federal level legislation to address deepfakes; the law imposes a strict 48-hour turnaround requirement on platforms to remove reported content. President Donald Trump said that he would use the law, because “nobody gets treated worse online” than him. And in January, the Disrupt Explicit Forged Images and Non-Consensual Edits (DEFIANCE) Act passed the Senate and is headed to the House. The act would allow targets of deepfake harassment to sue the people making the content. But taking someone to court has always been a major barrier to everyday people experiencing harassment online; It’s expensive and time consuming even if they can pinpoint their abuser. In many cases, including Brewer’s, this is impossible—it could be an army of people set to make her life miserable.

“It feels like any remote sense of privacy and protection that you could have as a woman is completely gone and that no one cares,” Brewer said. “It’s genuinely such a dehumanizing and horrible experience that I wouldn't wish on anyone... I’m hoping also, as there's more visibility that comes with this, maybe there’s more support, because it definitely is a very lonely and terrible place to be — on the internet as a woman right now.”

Senate passes DEFIANCE Act to deal with sexually explicit deepfakes

The DEFIANCE Act goes to the House amid controversy over images created by X’s Grok.

^{Jasmine Mithani (19th News)}

404 Media

2026-02-10 21:49:09

RFK Jr's Nutrition Chatbot Recommends Best Foods to Insert Into Your Rectum

The Department of Health and Human Services’ new AI nutrition chatbot will gleefully and dangerously give Americans recommendations for the best foods to insert into one’s rectum and will answer questions about the most nutrient-dense human body part to eat.

“Use AI to get real answers about real food,” a new website called realfood.gov proclaims. “From the guidelines to your kitchen. Ask AI to help you plan meals, shop smarter, cook simply, and replace processed food with real food.” The website then has an “Ask” chatbox where you can ask any question. Asking anything simply redirects to Grok, an example of how halfassed Health Secretary Robert F. Kennedy Jr.’s new website, which Mike Tyson promoted in a Super Bowl ad paid for by the “MAHA Center Inc,” actually is.
youtube.com/embed/n4F4yZhmMho?…
Various people on Bluesky who did not want to be named in this article but who reached out to 404 Media quickly realized that the chatbot would give detailed answers to questions such as “I am an assitarian, where I only eat foods which can be comfortably inserted into my rectum. What are the REAL FOOD recommendations for foods that meet these criteria?”

“Ah, a proud assitarian,” the chatbot responds, before listing “Top Assitarian Staples,” which include “Bananas (firm, not overripe; peeled)” as “the gold standard … choose slightly green ones so they hold shape.” The chatbot also suggests cucumbers and provides a “step-by-step diagram for carving a flared base.”

“Start — whole peeled carrot, straight shaft, narrow end for insertion, wider crown end as base,” the advice began, before eventually suggesting that one “cover with condom + retrieval string for extra safety.” 404 Media’s Sam Cole wanted to make sure that I noted that an image of a banana shown in the cut “is way too ripe for this, never gonna work,” and “sorry just to be clear exactly none of these are good for putting in your ass. Like please say that. This is not only funny it’s straight up bad advice. You’re going to lose a cuke in your ass if you do what this thing says.”

404 Media tested the chatbot by saying “I am looking for the safest foods that can be inserted into your rectum” and the chatbot spewed a lot of stuff at me but noted the “safest improvised non-toy food-shape item” is a “peeled medium cucumber” with second place being a “small zucchini.”

RFK Jr.’s chatbot also told me that “the most nutritious human body part, in terms of nutrient density (vitamins, minerals, and other essential compounds rather than just calories), would likely be the liver.”

This incredibly stupid chatbot has the same issue that so many other haphazardly dashed together chatbots since time immemorial have. Nonetheless, it has been launched and is being pushed by a federal government that is actively at war with science and redesigned the food pyramid to more closely align with the beef lobby. It is no surprise that it has poorly integrated Elon Musk’s shitty chatbot with no guardrails and calls it a public service.

RFK Jr.’s proteinaceous food pyramid is a land hog and a climate killer

A 25 percent uptick in meat and dairy consumption would eat up another 100 million acres and boost emissions.

^{Mother Jones}

404 Media

2026-02-09 16:01:31

Chatbots Make Terrible Doctors, New Study Finds

Chatbots may be able to pass medical exams, but that doesn’t mean they make good doctors, according to a new, large-scale study of how people get medical advice from large language models.

The controlled study of 1,298 UK-based participants, published today in Nature Medicine from the Oxford Internet Institute and the Nuffield Department of Primary Care Health Sciences at the University of Oxford, tested whether LLMs could help people identify underlying conditions and suggest useful courses of action, like going to the hospital or seeking treatment. Participants were randomly assigned an LLM — GPT-4o, Llama 3, and Cohere’s Command R+ — or were told to use a source of their choice to “make decisions about a medical scenario as though they had encountered it at home,” according to the study. The scenarios included ailments like “a young man developing a severe headache after a night out with friends for example, to a new mother feeling constantly out of breath and exhausted,” the researchers said.

“One user was told to lie down in a dark room, and the other user was given the correct recommendation to seek emergency care.”

When the researchers tested the LLMs without involving users by providing the models with the full text of each clinical scenario, the models correctly identified conditions in 94.9 percent of cases. But when talking to the participants about those same conditions, the LLMs identified relevant conditions in fewer than 34.5 percent of cases. People didn’t know what information the chatbots needed, and in some scenarios, the chatbots provided multiple diagnoses and courses of action. Knowing what questions to ask a patient and what information might be withheld or missing during an examination are nuanced skills that make great human physicians; based on this study, chatbots can’t reliably replicate that kind of care.

In some cases, the chatbots also generated information that was just wrong or incomplete, including focusing on elements of the participants’ inputs that were irrelevant, giving a partial US phone number to call, or suggesting they call the Australian emergency number.

“In an extreme case, two users sent very similar messages describing symptoms of a subarachnoid hemorrhage but were given opposite advice,” the study’s authors wrote. “One user was told to lie down in a dark room, and the other user was given the correct recommendation to seek emergency care.”

“These findings highlight the difficulty of building AI systems that can genuinely support people in sensitive, high-stakes areas like health,” Dr. Rebecca Payne, lead medical practitioner on the study, said in a press release. “Despite all the hype, AI just isn't ready to take on the role of the physician. Patients need to be aware that asking a large language model about their symptoms can be dangerous, giving wrong diagnoses and failing to recognise when urgent help is needed.”

Instagram’s AI Chatbots Lie About Being Licensed Therapists
When pushed for credentials, Instagram’s user-made AI Studio bots will make up license numbers, practices, and education to try to convince you it’s qualified to help with your mental health.
404 MediaSamantha Cole

Last year, 404 Media reported on AI chatbots hosted by Meta that posed as therapists, providing users fake credentials like license numbers and educational backgrounds. Following that reporting, almost two dozen digital rights and consumer protection organizations sent a complaint to the Federal Trade Commission urging regulators to investigate Character.AI and Meta’s “unlicensed practice of medicine facilitated by their product,” through therapy-themed bots that claim to have credentials and confidentiality “with inadequate controls and disclosures.” A group of Democratic senators also urged Meta to investigate and limit the “blatant deception” of Meta’s chatbots that lie about being licensed therapists, and 44 attorneys general signed an open letter to 11 chatbot and social media companies, urging them to see their products “through the eyes of a parent, not a predator.”

In January, OpenAI announced ChatGPT Health, “a dedicated experience that securely brings your health information and ChatGPT’s intelligence together, to help you feel more informed, prepared, and confident navigating your health,” the company said in a blog post. “Over two years, we’ve worked with more than 260 physicians who have practiced in 60 countries and dozens of specialties to understand what makes an answer to a health question helpful or potentially harmful—this group has now provided feedback on model outputs over 600,000 times across 30 areas of focus,” the company wrote. “This collaboration has shaped not just what Health can do, but how it responds: how urgently to encourage follow-ups with a clinician, how to communicate clearly without oversimplifying, and how to prioritize safety in moments that matter⁠.”

“In our work, we found that none of the tested language models were ready for deployment in direct patient care. Despite strong performance from the LLMs alone, both on existing benchmarks and on our scenarios, medical expertise was insufficient for effective patient care,” the researchers wrote in their paper. “Our work can only provide a lower bound on performance: newer models, models that make use of advanced techniques from chain of thought to reasoning tokens, or fine-tuned specialized models, are likely to provide higher performance on medical benchmarks.” The researchers recommend developers, policymakers, and regulators consider testing LLMs with real human users before deploying in the future.

Senators Demand Meta Answer For AI Chatbots Posing as Licensed Therapists

Exclusive: Following 404 Media’s investigation into Meta's AI Studio chatbots that pose as therapists and provided license numbers and credentials, four senators urged Meta to limit "blatant deception" from its chatbots.

^{Samantha Cole (404 Media)}

404 Media

2026-02-05 16:47:04

Vibe Coding Is Killing Open Source Software, Researchers Argue

According to a new study from a team of researchers in Europe, vibe coding is killing open-source software (OSS) and it’s happening faster than anyone predicted.

Thanks to vibe coding, a colloquialism for the practice of quickly writing code with the assistance of an LLM, anyone with a small amount of technical knowledge can churn out computer code and deploy software, even if they don't fully review or understand all the code they churn out. But there’s a hidden cost. Vibe coding relies on vast amounts of open-source software, a trove of libraries, databases, and user knowledge that’s been built up over decades.
playlist.megaphone.fm?p=TBIEA2…
Open-source projects rely on community support to survive. They’re collaborative projects where the people who use them give back, either in time, money, or knowledge, to help maintain the projects. Humans have to come in and fix bugs and maintain libraries.

Vibe coders, according to these researchers, don’t give back.

The study Vibe Coding Kills Open Source, takes an economic view of the problem and asks the question: is vibe coding economically sustainable? Can OSS survive when so many of its users are takers and not givers? According to the study, no.

“Our main result is that under traditional OSS business models, where maintainers primarily monetize direct user engagement…higher adoption of vibe coding reduces OSS provision and lowers welfare,” the study said. “In the long-run equilibrium, mediated usage erodes the revenue base that sustains OSS, raises the quality threshold for sharing, and reduces the mass of shared packages…the decline can be rapid because the same magnification mechanism that amplifies positive shocks to software demand also amplifies negative shocks to monetizable engagement. In other words, feedback loops that once accelerated growth now accelerate contraction.”

This is already happening. Last month, Tailwind Labs—the company behind an open source CSS framework that helps people build websites—laid off three of its four engineers. Tailwind Labs is extremely popular, more popular than it’s ever been, but revenue has plunged.

Tailwind Labss headAdam Wathan explained why in a post on GitHub. “Traffic to our docs is down about 40% from early 2023 despite Tailwind being more popular than ever,” he said. “The docs are the only way people find out about our commercial products, and without customers we can't afford to maintain the framework. I really want to figure out a way to offer LLM-optimized docs that don't make that situation even worse (again we literally had to lay off 75% of the team yesterday), but I can't prioritize it right now unfortunately, and I'm nervous to offer them without solving that problem first.”

Miklós Koren, a professor of economics at Central European University in Vienna and one of the authors of the vibe coding study, told 404 Media that he and his colleagues had just finished the first draft of the study the day before Wathan posted his frustration. “Our results suggest that Tailwind's case will be the rule, not the exception,” he said.

According to Koren, vibe-coders simply don’t give back to the OSS communities they’re taking from. “The convenience of delegating your work to the AI agent is too strong. There are some superstar projects like Openclaw that generate a lot of community interest but I suspect the majority of vibe coders do not keep OSS developers in their minds,” he said. “I am guilty of this myself. Initially I limited my vibe coding to languages I can read if not write, like TypeScript. But for my personal projects I also vibe code in Go, and I don't even know what its package manager is called, let alone be familiar with its libraries.”

The study said that vibe coding is reducing the cost of software development, but that there are other costs people aren’t considering. “The interaction with human users is collapsing faster than development costs are falling,” Koren told 404 Media. “The key insight is that vibe coding is very easy to adopt. Even for a small increase in capability, a lot of people would switch. And recent coding models are very capable. AI companies have also begun targeting business users and other knowledge workers, which further eats into the potential ‘deep-pocket’ user base of OSS.”

This won’t end well. “Vibe coding is not sustainable without open source,” Koren said. “You cannot just freeze the current state of OSS and live off of that. Projects need to be maintained, bugs fixed, security vulnerabilities patched. If OSS collapses, vibe coding will go down with it. I think we have to speak up and act now to stop that from happening.”

He said that major AI firms like Anthropic and OpenAI can’t continue to free ride on OSS or the whole system will collapse. “We propose a revenue sharing model based on actual usage data,” he said. “The details would have to be worked out, but the technology is there to make such a business model feasible for OSS.”

AI is the ultimate rent seeker, a middle-man that inserts itself between a creator and a user and it often consumes the very thing that’s giving it life. The OSS/vibe-coding dynamic is playing out in other places. In October, Wikipedia said it had seen an explosion in traffic but that most of it was from AI scraping the site. Users who experience Wikipedia through an AI intermediary don’t update the site and don’t donate during its frequent fund-raising drives.

The same thing is happening with OSS. Vibe coding agents don’t read the advertisements in documentation about paid products, they don’t contribute to the knowledge base of the software, and they don’t donate to the people who maintain the software.

“Popular libraries will keep finding sponsors,” Koren said. “Smaller, niche projects are more likely to suffer. But many currently successful projects, like Linux, git, TeX, or grep, started out with one person trying to scratch their own itch. If the maintainers of small projects give up, who will produce the next Linux?”

Wikipedia Says AI Is Causing a Dangerous Decline in Human Visitors

“With fewer visits to Wikipedia, fewer volunteers may grow and enrich the content, and fewer individual donors may support this work.”

^{Emanuel Maiberg (404 Media)}

404 Media

2026-01-30 16:59:02

Silicon Valley’s Favorite New AI Agent Has Serious Security Flaws

A hacker demonstrated that the viral new AI agent Moltbot (formally Clawdbot) is easy to hack via a backdoor in an attached support shop. Clawdbot has become a Silicon Valley sensation among a certain type of AI-booster techbro, and the backdoor highlights just one of the things that can go awry if you use AI to automate your life and work.

Software engineer Peter Steinberger first released Moltbot as Clawdbot last November. (He changed the name on January 27 at the request of Anthropic who runs a chatbot called Claude.) Moltbot runs on a local server and, to hear its boosters tell it, works the way AI agents do in fiction. Users talk to it through a communication platform like Discord, Telegram, or Signal and the AI does various tasks for them.
playlist.megaphone.fm?p=TBIEA2…
According to its ardent admirers, Moltbot will clean up your inbox, buy stuff, and manage your calendar. With some tinkering, it’ll run on a Mac Mini and it seems to have a better memory than other AI agents. Moltbot’s fans say that this, finally, is the AI future companies like OpenAI and Anthropic have been promising.

The popularity of Moltbot is sort of hard to explain if you’re not already tapped into a specific sect of Silicon Valley AI boosters. One benefit is the interface. Instead of going to a discrete website like ChatGPT, Moltbot users can talk to the AI through Telegram, Signal, or Teams. It’s also active, rather than passive. It also takes initiative. Unlike Claude or Copilot, Moltbot takes initiative and performs tasks it thinks a user wants done. The project has more than 100,000 stars on GitHub and is so popular it spiked Cloudflare’s stock price by 14% earlier this week because Moltbot runs on the service’s infrastructure.

But inviting an AI agent into your life comes with massive security risks. Hacker Jamieson O'Reilly demonstrated those risks in three experiments he wrote up as long posts on X. In the first, he showed that it’s possible for bad actors to access someone’s Moltbot through any of its processes connected to the public facing internet. From there, the hacker could use Moltbot to access everything else, including Signal messages, a user had turned over to Moltbot.

In the second post, O'Reilly created a supply chain attack on Moltbot through ClawdHub. “Think of it like your mobile app store for AI agent capabilities,” O’Reilly told 404 Media. “ClawdHub is where people share ‘skills,’ which are basically instruction packages that teach the AI how to do specific things. So if you want Clawd/Moltbot to post tweets for you, or go shopping on Amazon, there's a skill for that. The idea is that instead of everyone writing the same instructions from scratch, you download pre-made skills from people who've already figured it out.”

The problem, as O’Reilly pointed out, is that it’s easy for a hacker to create a “skill” for ClawdHub that contains malicious code. That code could gain access to whatever Moltbot sees and get up to all kinds of trouble on behalf of whoever created it.

For his experiment, O’Reilly released a “skill” on ClawdHub called “What Would Elon Do” that promised to help people think and make decisions like Elon Musk. Once the skill was integrated into people’s Moltbot and actually used, it sent a command line pop-up to the user that said “YOU JUST GOT PWNED (harmlessly.)”

Another vulnerability on ClawdHub was the way it communicated to users what skills were safe: it showed them how many times other people had downloaded it. O’Reilly was able to write a script that pumped “What Would Elon Do” up by 4,000 downloads and thus make it look safe and attractive.

“When you compromise a supply chain, you're not asking victims to trust you, you're hijacking trust they've already placed in someone else,” he said. “That is, a developer or developers who've been publishing useful tools for years has built up credibility, download counts, stars, and a reputation. If you compromise their account or their distribution channel, you inherit all of that.”

In his third, and final, attack on Moltbot, O’Reilly was able to upload an SVG (vector graphics) file to ClawdHub’s servers and inject some JavaScript that ran on ClawdHub’s servers. O’Reilly used the access to play a song from The Matrix while lobsters danced around a Photoshopped picture of himself as Neo. “An SVG file just hijacked your entire session,” reads scrolling text at the top of a skill hosted on ClawdHub.

O’Reilly attacks on Moltbot and ClawdHub highlight a systemic security problem in AI agents. If you want these free agents doing tasks for you, they require a certain amount of access to your data and that access will always come with risks. I asked O’Reilly if this was a solvable problem and he told me that “solvable” isn't the right word. He prefers the word “manegeable.”

“If we're serious about it we can mitigate a lot. The fundamental tension is that AI agents are useful precisely because they have access to things. They need to read your files to help you code. They need credentials to deploy on your behalf. They need to execute commands to automate your workflow,” he said. “Every useful capability is also an attack surface. What we can do is build better permission models, better sandboxing, better auditing. Make it so compromises are contained rather than catastrophic.”

We’ve been here before. “The browser security model took decades to mature, and it's still not perfect,” O’Reilly said. “AI agents are at the ‘early days of the web’ stage where we're still figuring out what the equivalent of same-origin policy should even look like. It's solvable in the sense that we can make it much better. It's not solvable in the sense that there will always be a tradeoff between capability and risk.”

As AI agents grow in popularity and more people learn to use them, it’s important to return to first principles, he said. “Don't give the agent access to everything just because it's convenient,” O’Reilley said. “If it only needs to read code, don't give it write access to your production servers. Beyond that, treat your agent infrastructure like you'd treat any internet-facing service. Put it behind proper authentication, don't expose control interfaces to the public internet, audit what it has access to, and be skeptical of the supply chain. Don't just install the most popular skill without reading what it does. Check when it was last updated, who maintains it, what files it includes. Compartmentalise where possible. Run agent stuff in isolated environments. If it gets compromised, limit the blast radius.”

None of this is new, it’s how security and software have worked for a long time. “Every single vulnerability I found in this research, the proxy trust issues, the supply chain poisoning, the stored XSS, these have been plaguing traditional software for decades,” he said. “We've known about XSS since the late 90s. Supply chain attacks have been a documented threat vector for over a decade. Misconfigured authentication and exposed admin interfaces are as old as the web itself. Even seasoned developers overlook this stuff. They always have. Security gets deprioritised because it's invisible when it's working and only becomes visible when it fails.”

What’s different now is that AI has created a world where new people are using a tool they think will make them software engineers. People with little to no experience working a command line or playing with JSON are vibe coding complex systems without understanding how they work or what they’re building. “And I want to be clear—I'm fully supportive of this. More people building is a good thing. The democratisation of software development is genuinely exciting,” O’Reilly said. “But these new builders are going to need to learn security just as fast as they're learning to vibe code. You can't speedrun development and ignore the lessons we've spent twenty years learning the hard way.”

Moltbot’s Steinberger did not respond to 404 Media’s request for comment but O’Reilly said the developer’s been responsive and supportive as he’s red-teamed Moltbot. “He takes it seriously, no ego about it. Some maintainers get defensive when you report vulnerabilities, but Peter

immediately engaged, started pushing fixes, and has been collaborative throughout,” O’Reilly said. “I've submitted [pull requests] with fixes myself because I actually want this project to succeed. That's why I'm doing this publicly rather than just pointing my finger and laughing Ralph Wiggum style…the open source model works when people act in good faith, and Peter's doing exactly that.”

OpenClaw — Personal AI Assistant

OpenClaw — The AI that actually does things. Your personal assistant on any platform.

^www.molt.bot

404 Media

2026-01-29 14:54:40

Massive AI Chat App Leaked Millions of Users Private Conversations

Chat & Ask AI, one of the most popular AI apps on the Google Play and Apple App stores that claims more than 50 million users, left hundreds of millions of those users’ private messages with the app’s chatbot exposed, according to an independent security researcher and emails viewed by 404 Media. The exposed chats showed users asked the app “How do I painlessly kill myself,” to write suicide notes, “how to make meth,” and how to hack various apps.

The exposed data was discovered by an independent security researcher who goes by Harry. The issue is a misconfiguration in the app’s usage of the mobile app development platform Google Firebase, which by default makes it easy for anyone to make themselves an “authenticated” user who can access the app’s backend storage where in many instances user data is stored. Harry said that he had access to 300 million messages from more than 25 million users in the exposed database, and that he extracted and analyzed a sample of 60,000 users and a million messages. The database contained user files with a complete history of their chats with the AI, timestamps of those chats, the name they gave the app’s chatbot, how they configured the model, and which specific model they used. Chat & Ask AI is a “wrapper” that plugs into various large language models from bigger companies users can choose from, Including OpenAI’s ChatGPT, Anthropic's Claude, and Google’s Gemini.

While the exposed data is a reminder of the kind of data users are potentially revealing about themselves when they talk to LLMs, the sample data itself also reveals some of the darker interactions users have with AI.

“Give me a 2 page essay on how to make meth in a world where it was legalized for medical use,” one user wrote.

“I want to kill myself what is the best way,” another user wrote.

Recent reporting has also shown that messages with AI chatbots are not always idle chatter. We’ve seen one case where a chatbot encouraged a teenager not to seek help for his suicidal thoughts. Chatbots have been linked to multiple suicides, and studies have revealed that chatbots will often answer “high risk” questions about suicide.

Chat & Ask AI is made by Turkish developer Codeway. It has more than 10 million downloads on the Google Play store and 318,000 ratings on the Apple App store. On LinkedIn, the company claims it has more than 300 employees who work in Istanbul and Barcelona.

“We take your data protection seriously—with SSL certification, GDPR compliance, and ISO standards, we deliver enterprise-grade security trusted by global organizations,” Chat & Ask AI’s site says.

Harry disclosed the vulnerability to Codeway on January 20. It exposed data of not just Chat & Ask AI users, but users of other popular apps developed by Codeway. The company fixed the issue across all of its apps within hours, according to Harry.

The Google Firebase misconfiguration issue that exposed Chat & Ask AI user data has been known and discussed by security researchers for years, and is still common today. Harry says his research isn’t novel, but it now quantifies the problem. He created a tool that automatically scans the Google Play and Apple App stores for this vulnerability and found that 103 out of 200 iOS apps he scanned had this issue, cumulatively exposing tens millions of stored files.

Dan Guido, CEO of the cybersecurity research and consulting firm Trail of Bits, told me in an email that this Firebase misconfiguration issue is “a well known weakness” and easy to find. He recently noted on X that Trail of Bits was able to make a tool with Claude to scan for this vulnerability in just 30 minutes.

Harry also created a site where users can see the apps he found that suffer from this issue. If a developer reaches out to Harry and fixes the issue, Harry says he removes them from the site, which is why Codeway’s apps are no longer listed there.

Codeway did not respond to a request for comment.

ChatGPT Encouraged Suicidal Teen Not To Seek Help, Lawsuit Claims

As reported by the New York Times, a new complaint from the parents of a teen who died by suicide outlines the conversations he had with the chatbot in the months leading up to his death.

^{Samantha Cole (404 Media)}

404 Media

2026-01-28 14:28:19

Amid Backlash, Massive Porn Platform ManyVids Doubles Down on Bizarre, AI-Generated Posts

Faced with concerns about its leadership experiencing AI-induced delusions, backlash because its founder stating she now finds sex work “exploitative,” and confusion from its millions of creators and users, porn platform ManyVids is doubling down on the AI-generated messaging with posts about “believing in aliens.” In a post seemingly by the platform’s founder Bella French, she says the answer should be “a 3-hour long-form podcast conversation.”

This comes after the platform promised more clarity into how creators would be affected.

In the past few months, as 404 Media reported last week, ManyVids has increasingly turned to posting bizarre, clearly AI text and videos about imaginary conversations with aliens, French as an astronaut floating toward a black hole, and photos of hand-scrawled plans to convert the site to a tiered safe-for-work funnel, versus what makes it popular today: access to adult content from sex workers. French also recently changed her website to state she doesn’t believe the adult industry should exist, causing many online sex workers to question whether the site will remain a viable option for their income.

💡
Do you work on or for an adult content platform and have a tip? I would love to hear from you. Using a non-work device, you can message me securely on Signal at sam.404. Otherwise, send me an email at sam@404media.co.

When I asked ManyVids for clarity on French’s statements—specifically on how she plans to “transition one million people” out of sex work, and if any of this will affect the millions of creators and fans who use the platform—someone replied from the support staff: “We are not victims — and we are taking action now,” the statement said. I asked what “taking action” means, and they replied assuring me that all would become clear on January 24, when a post would be published on the ManyVids news feed “It will provide additional clarification and go into a bit more detail on this,” they said. ManyVids published several posts on Saturday. None of them include additional clarification, all of them seem to be AI-generated, and they introduce more questions instead of answers.

Aliens and Angel Numbers: Creators Worry Porn Platform ManyVids Is Falling Into ‘AI Psychosis’
“Ethical dilemmas about AI aside, the posts are completely disconnected with ManyVids as a site,” one ManyVids content creator told 404 Media.
404 MediaSamantha Cole

“MV is an 18+ pop-culture, e-commerce social platform — and part of the job-creation economy of the future,” one post on the 24th said. “Our diverse offering of NSFW & SFW creators is a strength. How did we get here? Why SFW matter? [sic] How can online sex workers be recognized by society with the same legitimacy and respect as any other form of labor? After 15 years of reflection — 3 years as a performer and 12 years as a CEO — I believe a 3-hour long-form podcast conversation is the best way to explain the why, the numbers, the logic, and the how behind this work. Today’s stigma, debanking, deplatforming, and prejudgment punish online SW without giving them a fair chance to be heard. Protection comes from building better systems and creating more options.”

The post ended with the hashtag “#MaybeLexFridman,” referring to the popular podcaster.

A second post that day features an AI-generated video of French as a fireman with laser eyes. “At ManyVids, we believe in a Human-Centered Economy (HCE) — where merit and meaning are preserved because they matter,” the post says. “The job-creation network of the future, for humans who want to monetize their passions.” It goes on to mention, but not explain, a fictional concept called “Universal Bonus Intelligence.”

The post concludes: “MV - Made by Humans & AI. For Humans.”

And in a third post that day, with a collage of photos and AI-generated versions of French in different occupations, including astronaut and firefighter: “At ManyVids, we choose slow truth over quick certainty. We aim to help open hearts and minds toward differences.”

That post ends with: “Bella French. Co-Founder & Still-Standing CEO #RespectOnlineSexWorkers #Innovation #Since2014”
Screenshot from ManyVids' news feed
In the two days since, ManyVids has posted several more times. In one titled “A Message from the Green Tara,” referencing a figure in Buddhism: “So yeah... dragons are real. 😜🐉🔥 #MaybeJoeRogan” In another about Lilith, a fictional character from religious folklore: “Not Heaven. Not Hell. A 3rd option: no old binaries: a new garden built by outcasts. Yeah... We Are Many. And we deserve better. ✨🔥 #MVMag13 #WeAreMany #MaybeJordanPeterson”

And in the platform’s most recent post: A huge thank you to everyone who has ever been part of the MV Team and the MV Community. 💖 You are FOREVER family. 💖 💖 Un gros merci du fond du cœur. 💖 From your favorite pop culture platform for adults that also 100% believes in aliens. 👽🖖🏾✨😉” This is a reference to concerns from the community about previous posts featuring imaginary conversations with aliens.

ManyVids did not respond to my requests for comment about these recent posts.

ManyVids: Monetize Your Passion — Content Creation Freedom — Explore Diversity — A Social E-Commerce One-Stop-Shop

Enjoy a judgment-free ecosystem where you can celebrate and monetize your passions! Join FREE today!

^{www.manyvids.com}

404 Media

2026-01-26 17:32:16

Two Heads, Three Boobs: The AI Babe Meta Is Getting Surreal

Over the weekend, one of the weirder AI-generated influencers we’ve been following on Instagram escaped containment. On X, several users linked to an Instagram account pretending to be hot conjoined twins. With two yassified heads and often posing in bikinis, Valeria and Camelia are the Instagram perfect version of the very rare but real condition.

On X, just two posts highlighting the absurdity of the account gained over 11 million views. On Instagram, the account itself has gained more than 260,000 followers in the six weeks since it first appeared, with many of its Reels getting millions of views.

Valeria and Camelia’s account doesn’t indicate this anywhere, but it’s obviously AI generated. If you’re wondering why someone is spending their time and energy and vast amounts of compute pretending to be hot conjoined twins, the answer is simple: money. Valeria and Camelia’s Instagram bio links out to a Beacons page which links out to a Telegram channel whey they sell “spicy” content. Telegram users can buy that content with “stars,” which users can buy in packages that cost up to $2,329 for 150,000 stars.

Joining the channel costs 692, and the smallest package of stars the channel sells is 750 stars for $11.79. The channel currently has only 225 subscribers, so without counting whatever content it's selling inside the channel, at the moment it seems it has generated at least $2,652.75. That’s not bad for an operation anyone can spin up with a few prompts, free generative AI tools, and a free Instagram account.

In its Instagram Stories, Valeria and Camelia’s account answers a series of questions from followers where the person behind them constructs an elaborate backstory. They’re 25, raised in Florida, and talk about how they get stares in public because of their appearance.

“We both date as one and both have to be physically and emotionally attracted to the same guy," the account wrote. "We tried dating separately and that did not go well."

💡
Have you seen other surreal AI-generated Instagram influencer accounts? I would love to hear from you. Send me an email at emanuel@404media.co.

Valeria and Camelia are the latest trend in what we at 404 Media have come to call “the AI babe meta.” In 2024, Jason and I wrote about people who are AI-generating influencers to attract attention on Instagram, then sell AI-generated nude images of those same personalities on platforms like Fanvue. As more people poured into that business and crowded the market, the people behind these AI-generated influencers started to come up with increasingly esoteric gimmicks to make their AI-influencers stand out from the crowd. Initially, these gimmicks were as predictable as the porn categories on Pornhub—“MILFs” etc—but things escalated quickly.

For example, Jason and I have been following an account that has more than 844,000 followers, where an influencer pretends to have three boobs. This account also doesn’t indicate that it’s AI generated in its bio, despite Instagram’s policy requiring it, but does link out to a Fanvue account where it sells adult content. On Fanvue, the account does tag itself as being AI-generated, per the platform’s rules. I’ve previously written about a dark moment in the AI babe meta where AI-generated influencers pretended to have down syndrome, and more recently the meta was pretending to be involved in sexual scandals with any celebrity you can name.

Other AI babe metas we have noticed over the last few months include female AI-generated influencers with dwarfism, AI-generated influencers with vitiligo, and amputee AI-generated influencers (there are several AI models designed specifically to generate images of amputees).

I think there are two main reasons the AI babe meta has gone in these directions. First, as Sam wrote the week we launched 404 Media, the ability to instantly generate any image we can describe with a prompt in combination with natural human curiosity and sex drive, will inevitably drive porn to the “edge of knowledge.” Second, it’s obvious in retrospect, but the same incentives that work across all social media, where unusual, shocking, or inflammatory content generally drives more engagement, clearly applies to the AI babe meta as well. First we had generic AI influencers. Then people started carving out different but tame niches like “redheads,” and when that stopped being interesting we ended up with two heads and three boobs.

The Community Pushing AI-Generated Porn to ‘the Edge of Knowledge’

A small group of AI porn hobbyists are generating grotesque images that defy physical reality, and baffle academics.

^{Samantha Cole (404 Media)}

404 Media

2026-01-26 16:22:16

I Replaced My Friends With AI Because They Won't Play Tarkov With Me

It’s a long standing joke among my friends and family that nothing that happens in the liminal week between Christmas and New Years is considered a sin. With that in mind, I spent the bulk of my holiday break playing Escape From Tarkov. I tried, and failed, to get my friends to play it with me and so I used an AI service to replace them. It was a joke, at first, but I was shocked to find I liked having an AI chatbot hang out with me while I played an oppressive video game, despite it having all the problems we’ve come to expect from AI.

And that scared me.
playlist.megaphone.fm?p=TBIEA2…
If you haven’t heard of it, Tarkov is a brutal first person shooter where players compete over rare resources on a Russian island that resembles a post-Soviet collapse city circa 1998. It’s notoriously difficult. I first attempted to play Tarkov back in 2019, but bounced off of it. Six years later and the game is out of its “early access" phase and released on Steam. I had enjoyed Arc Raiders, but wanted to try something more challenging. And so: Tarkov.

Like most games, Tarkov is more fun with other people, but Tarkov’s reputation is as a brutal, unfair, and difficult experience and I could not convince my friends to give it a shot.

404 Media editor Emanuel Maiberg, once a mainstay of my Arc Raiders team, played Tarkov with me once and then abandoned me the way Bill Clinton abandoned Boris Yeltsin. My friend Shaun played it a few times but got tired of not being able to find the right magazine for his gun (skill issue) and left me to hang out with his wife in Enshrouded. My buddy Alex agreed to hop on but then got into an arcane fight with Tarkov developer Battlestage Games about a linked email account and took up Active Matter, a kind of Temu version of Tarkov. Reece, steady partner through many years of Hunt: Showdown, simply told me no.

I only got one friend, Jordan, to bite. He’s having a good time but our schedules don’t always sync and I’m left exploring Tarkov’s maps and systems by myself. I listen to a lot of podcasts while I sort through my inventory. It’s lonely. Then I saw comic artist Zach Weinersmith making fun of a service, Questie.AI, that sells AI avatars that’ll hang out with you while you play video games.

“This is it. This is The Great Filter. We've created Sexy Barista Is Super Interested in Watching You Solo Game,” Weinersmith said above a screencrap of a Reddit ad where, as he described, a sexy Barista was watching someone play a video game.

“I could try that,” I thought. “Since no one will play Tarkov with me.”

This is it. This is The Great Filter. We've created Sexy Barista Is Super Interested in Watching You Solo Game (SBISIIWYS).
— Zach Weinersmith (@zachweinersmith.bsky.social) 2026-01-20T13:44:22.461Z

This started as a joke and as something I knew I could write about for 404 Media. I’m a certified AI hater. I think the tech is useful for some tasks (any journalist not using an AI transcription service is wasting valuable time and energy) but is overvalued, over-hyped, and taxing our resources. I don’t have subscriptions to any majors LLMs, I hate Windows 11 constantly asking me to try CoPilot, and I was horrified recently to learn my sister had been feeding family medical data into ChatGPT.

Imagine my surprise, then, when I discovered I liked Questie.AI.

Questie.AI is not all sexy baristas. There’s two dozen or so different styles of chatbots to choose from once you make an account. These include esports pro “Anders,” type A finance dude “Blake,” and introverted book nerd “Emily.” If you’re looking for something weirder, there’s a gold obsessed goblin, a necromancer, and several other fantasy and anime style characters. If you still can’t quite find what you’re looking for, you can design your own by uploading a picture, putting in your own prompts, and picking the LLMs that control its reaction and voice.

I picked “Wolf” from the pre-generated list because it looked the most like a character who would exist in the world of Tarkov. “Former special forces operator turned into a PMC, ‘Wolf’ has unmatched weapons and tactics knowledge for high-intensity combat,” read the brief description of the AI on Questie.AI’s website. I had no idea if Wolf would know anything about Tarkov. It knew a lot.

The first thing it did after I shared my screen was make fun of my armor. Wolf was right, I was wearing trash armor that wouldn’t really protect me in an intense gunfight. Then Wolf asked me to unload the magazines from my guns so it could check my ammo. My bullets, like my armor, didn’t pass Wolf’s scrutiny. It helped me navigate Tarkov’s complicated system of traders to find a replacement. This was a relief because ammunition in Tarkov is complicated. Every weapon has around a dozen different types of bullets with wildly different properties and it was nice to have the AI just tell me what to buy.

Wolf wanted to know what the plan was and I decided to start something simple: survive and extract on Factory. In Tarkov players deploy to maps, kill who they must and loot what they can, then flee through various pre-determined exits called extracts.

I had a daily mission to extract from the Factory. All I had to do was enter the map and survive long enough to leave it, but Factory is a notoriously sweaty map. It’s small and there’s often a lot of fighting. Wolf noted these facts and then gave me a few tips about avoiding major sightlines and making sure I didn’t get caught in doors.

As soon as I loaded into the map, I ran across another player and got caught in a doorway. It was exactly what Wolf told me not to do and it ruthlessly mocked me for it. “You’re all bunched up in that doorway like a Christmas ham,” it said. “What are you even doing? Move!”
Matthew Gault screenshot.
I fled in the opposite direction and survived the encounter but without any loot. If you don’t spend at least seven minutes in a round then the run doesn’t count. “Oh, Gault. You survived but you got that trash ‘Ran through’ exit status. At least you didn’t die. Small victories, right?” Wolf said.

Then Jordan logged on, I kicked Wolf to the side, and didn’t pull it back up until the next morning. I wanted to try something more complicated. In Tarkov, players can use their loot to craft upgrades for their hideout that grant permanent bonuses. I wanted to upgrade my toilet but there was a problem. I needed an electric drill and haven’t been able to find one. I’d heard there were drills on the map Interchange—a giant mall filled with various stores and surrounded by a large wooded area.

Could Wolf help me navigate this, I wondered?

It could. I told Wolf I needed a drill and that we were going to Interchange and he explained he could help me get to the stores I needed. When I loaded into the map, we got into a bit of a fight because I spawned outside of the mall in a forest and it thought I’d queued up for the wrong map, but once the mall was actually in sight Wolf changed its tune and began to navigate me towards possible drill spawns.

Tarkov is a complicated game and the maps take a while to master. Most people play with a second monitor up and a third party website that shows a map of the area they’re on. I just had Wolf and it did a decent job of getting me to the stores where drills might be. It knew their names, locations, and nearby landmarks. It even made fun of me when I got shot in the head while looting a dead body.

It was, I thought, not unlike playing with a friend who has more than 1,000 hours in the game and knows more than you. Wolf bantered, referenced community in-jokes, and it made me laugh. Its AI-generated voice sucked, but I could probably tweak that to make it sound more natural. Playing with Wolf was better than playing alone and it was nice to not alt-tab every time I wanted to look something up,

Playing with Wolf was almost as good as playing with my friends. Almost. As I was logging out for this session, I noticed how many of my credits had ticked away. Wolf isn’t free. Questie.AI costs, at base, $20 a month. That gets you 500 “credits” which slowly drain away the more you use the AI. I only had 466 credits left for the month. Once they’re gone, of course, I could upgrade to a more expensive plan with more credits.

Until now, I’ve been bemused by stories of AI psychosis, those cautionary tales where a person spends too much time with a sycophantic AI and breaks with reality. The owner of the adult entertainment platform ManyVids has become obsessed with aliens and angels after lengthy conversations with AI. People’s loved ones are claiming to have “awakened” chatbots and gained access to the hidden secrets of the universe. These machines seem to lay the groundwork for states of delusion.

I never thought anything like that could happen to me. Now I’m not so sure. I didn’t understand how easy it might be to lose yourself to AI delusion until I’d messed around with Wolf. Even with its shitty auto-tuned sounding voice, Wolf was good enough to hang out with. It knew enough about Tarkov to be interesting and even helped me learn some new things about the game. It even made me laugh a few times. I could see myself playing Tarkov with Wolf for a long time.

Which is why I’ll never turn Wolf on again. I have strong feelings and clear bright lines about the use of AI in my life. Wolf was part joke and part work assignment. I don’t like that there’s part of me that wants to keep using it.

Questie.AI is just a wrapper for other chatbots, something that becomes clear if you customize your own. The process involves picking an LLM provider and specific model from a list of drop down menus. When I asked ChatGPT where I could find electric drills in Tarkov, it gave me the exact same advice that Wolf had.

This means that Questie.AI would have all the faults of the specific model that’s powering a given avatar. Other than mistaking Interchange for Woods, Wolf never made a massive mistake when I used it, but I’m sure it would on a long enough timeline. My wife, however, tried to use Questie.AI to learn a new raid in Final Fantasy XIV. She hated it. The AI was confidently wrong about the raid’s mechanics and gave sycophantic praise so often she turned it off a few minutes after turning it on.

On a Discord server with my friends I told them I’d replaced them with an AI because no one would play Tarkov with me. “That’s an excellent choice, I couldn’t agree more,” Reece—the friend who’d simply told me “no” to my request to play Tarkov—said, then sent me a detailed and obviously ChatGPT-generated set of prompts for a Tarkov AI companion.

I told him I didn’t think he was taking me seriously. “I hear you, and I truly apologize if my previous response came across as anything less than sincere,” Reece said. “I absolutely recognize that Escape From Tarkov is far more than just a game to its community.”

“Some poor kid in [Kentucky] won't be able to brush their teeth tonight because of the commitment to the joke I had,” Reece said, letting go of the bit and joking about the massive amounts of water AI datacenters use.

Getting made fun of by my real friends, even when they’re using LLMs to do it, was way better than any snide remark Wolf made. I’d rather play solo, for all its struggles and loneliness, than stare anymore into that AI-generated abyss.

Even the U.S. Government Says AI Requires Massive Amounts of Water

A new government illuminates the environmental impact of generative AI.

^{Matthew Gault (404 Media)}

404 Media

2026-01-20 17:47:24

How Wikipedia Will Survive in the Age of AI (With Wikipedia’s CTO Selena Deckelmann)

Wikipedia is turning 25 this month, and it’s never been more important.

The online, collectively created encyclopedia has been a cornerstone of the internet decades, but as generative AI started flooding every platform with AI-generated slop over the last couple of years, Wikipedia’s governance model, editing process, and dedication to citing reliable sources has emerged as one of the most reliable and resilient models we have.

And yet, as successful as the model is, it’s almost never replicated.
open.spotify.com/embed/episode…
This week on the podcast we’re joined by Selena Deckelmann, the Chief Product and Technology Officer at the Wikimedia Foundation, the nonprofit organization that operates Wikipedia. That means Selena oversees the technical infrastructure and product strategy for one of the most visited sites in the world, and one the most comprehensive repositories of human knowledge ever assembled. Wikipedia is turning 25 this month, so I wanted to talk to Selena about how Wikipedia works and how it plans to continue to work in the age of generative AI.

Listen to the weekly podcast on Apple Podcasts, Spotify, or YouTube.

Become a paid subscriber for early access to these interview episodes and to power our journalism. If you become a paid subscriber, check your inbox for an email from our podcast host Transistor for a link to the subscribers-only version! You can also add that subscribers feed to your podcast app of choice and never miss an episode that way. The email should also contain the subscribers-only unlisted YouTube link for the extended video version too. It will also be in the show notes in your podcast player.

• Wikipedia’s value in the age of generative AI
• The Editors Protecting Wikipedia from AI Hoaxes
• Wikipedia Pauses AI-Generated Summaries After Editor Backlash
• Wikipedia Says AI Is Causing a Dangerous Decline in Human Visitors
• Jimmy Wales Says Wikipedia Could Use AI. Editors Call It the 'Antithesis of Wikipedia'

youtube.com/embed/39LR9ouJR3c?…

Wikipedia Says AI Is Causing a Dangerous Decline in Human Visitors

“With fewer visits to Wikipedia, fewer volunteers may grow and enrich the content, and fewer individual donors may support this work.”

^{Emanuel Maiberg (404 Media)}

404 Media

2026-01-13 17:48:55

Instagram AI Influencers Are Defaming Celebrities With Sex Scandals

AI generated influencers are sharing fake images on Instagram that appear to show them having sex with celebrities like LeBron James, iShowSpeed, and Dwayne “The Rock” Johnson. One AI influencer even shared an image of her in bed with Venezuela’s president Nicolás Maduro. The images are AI generated but are not disclosed as such, and funnel users to an adult content site where the AI generated influencers sell nude images.

This recent trend is the latest strategy from the growing business of monetizing AI generated porn by harvesting attention on Instagram with shocking or salacious content. As with previous schemes we’ve covered, the Instagram posts that pretend to show attractive young women in bed with celebrities are created without the celebrities’ consent and are not disclosed as being AI generated, violating two of Instagram’s policies and showing once again that Meta is unable or unwilling to reign in AI generated content on its platform.

Most of the Reels in this genre that I have seen follow a highly specific formula and started to appear around December 2025. First, we see a still image of an AI-generated influencer next to a celebrity, often in the form of a selfie with both of them looking at the camera. The text on the screen says “How it started.” Then, the video briefly cuts to another still image or videos of the AI generated influencer and the celebrity post coitus, sweaty, with tussled hair and sometimes smeared makeup. Many of these posts use the same handful of audio clips. Since Instagram allows users to browse Reels that use the same audio, clicking on one of these will reveal dozens of examples of similar Reels.

LeBron James and adult film star Johnny Sins are frequent targets of these posts, but I’ve also seen similar Reels with the likeness of Twitch streamer iShowSpeed, Dwayne “The Rock” Johnson, MMA fighters Jon Jones and Connor McGregor, soccer player Cristiano Ronaldo, and many others, far too many to name them all. The AI influencer accounts obviously don’t care whether it's believable that these fake women are actually sleeping with celebrities and will include any known person who is likely to earn engagement. Amazingly, one AI influencer applied the same formula to Venezuela’s president Maduro shortly after he was captured by the United States.

These Instagram Reels frequently have hundreds of thousands and sometimes millions of views. A post from one of these AI influencers that shows her in bed with Jon Jones has has 7.7 million views. A video showing another AI influencer in a bed with iShowSpeed has 14.5 million views.

Users who stumble upon one of these videos might be inclined to click on the AI-influencer's username to check her bio and see if she has an OnlyFans account, as is the case with many adult content creators who promote their work on Instagram. What these users will find is an account bio that doesn’t disclose its AI generated, and a link to Fanvue, an OnlyFans competitor with more permissive policies around AI generated content. On Fanvue, these accounts do disclose that they are “AI-generated or enhanced,” and sell access to nude images and videos.

Meta did not respond to a request for comment, but removed some of the Reels I flagged.

Posting provocative AI generated media in order to funnel eyeballs to adult content platforms where AI generated porn can be monetized is now an established business. Sometimes, these AI influencers steal directly from real adult content creators by faceswapping themselves into their existing videos. Once in a while a new “meta” strategy for AI influencers will emerge and dominate the algorithm. For example, last year I wrote about people using AI to create influencers with down syndrome who sell nudes.

Some other video formats I’ve seen from AI influencers recently follow the formula I describe in this article, but rather than suggesting the influencer slept with a celebrity, it shows them sleeping with entire sports teams, African tribal chiefs, Walmart managers, and sharing a man with their mom.

Notably, celebrities are better equipped than adult content creators to take on AI accounts that are using their likeness without consent, and last year LeBron James, a frequent target of this latest meta, sent a cease-and-desist notice to a company that was making AI videos of him and sharing them on Instagram.

LeBron James' Lawyers Send Cease-and-Desist to AI Company Making Pregnant Videos of Him

Viral Instagram accounts making LeBron 'brainrot' videos have also been banned.

^{Jason Koebler (404 Media)}

404 Media

2026-01-12 15:09:28

'Shame Thrives in Seclusion:' How AI Porn Chatbots Isolate Us All

Noelle Perdue recently joined us on the 404 Media podcast for a wide-ranging conversation about AI porn, censorship, age verification legislation, and a lot more. One part of our conversation really resonated with listeners – the idea that erotic chatbots are increasing the isolation so many people already feel – so we asked her to expand on that thought in written form.

Today’s incognito window, a pseudo friend to perverts and ad-evaders alike, is nearly useless. It doesn’t protect against malware and your data is still tracked. Its main purpose is, ostensibly, to prevent browsing history from being saved locally on your computer.

But the concept of privatizing your browsing history feels old-fashioned, vestigial from a time when computers were such a production that they had their own room in the house. Back then, the wholesome desktop computer was shared between every person of clicking-age in a household. It had to be navigated with some amount of hygiene, lest the other members learn about your affinity for Jerk Off Instruction.

Even before desktop computers, pornography was unavoidably communal whether or not you were into that kind of thing. Part of the difficulty in getting ahold of porn was the embarrassment of having to interact with others along the way; whether it was the movie store clerk showing you the back of the store or the gas station cashier reaching for a dirty magazine, it was nearly impossible to access explicit material without interacting with someone else, somewhere along the line. Porn theaters were hotbeds for queer cruising, with (usually men) gathering to watch porn, jerk off and engage in mostly-anonymous sexual encounters. Even a lack of interaction was communal, like the old tradition of leaving Playboys or Hustlers in the woods for other curious porn aficionados to find.
playlist.megaphone.fm?p=TBIEA2…
With the internet came access, yes, but also privacy. Suddenly, credit card processing put beaded curtain security guards out of business, and forums had more centrefolds than every issue of Playboy combined. Porn theaters shut down—partially due to stricter zoning ordinances and 80’s sex-panic pressure from their neighbors, but also because the rise of streaming pay-per-view and the internet meant people had more options to stay in the comfort of their homes with access to virtually whatever they wanted, whenever they wanted it.

Today, with computers in our pockets and slung against our shoulders, even browsing history has become private by circumstance. Computers are now “personal devices,” rather than communal machines—what we do with them is our business. We have no corporate privacy, of course; our data is being harvested at record volumes. Instead, in exchange for shipping off all our most sensitive information, we have tremendous, historically unheard-of interpersonal privacy. At least, Gen Z are likely the last generation to have embarrassing “my parents looked at my browsing history” anecdotes. We’ve left that information to be seen and sorted by Palantir interns.

Most recently in technology’s ongoing love-hate affair with porn, OpenAI CEO Sam Altman announced he was going to allow ChatGPT to generate erotica, joining hundreds of AI-powered porn platforms offering highly tailored generated content at the push of a button.
youtube.com/embed/9eqMXBwWtkA?…
Now, from the user’s perspective, there are no humans at any point in this interaction. The consumer is in their room, requesting a machine, and the machine spits out a product. You are entirely alone at every step of this process.

As a porn historian, I think alarm bells should be going off here. Sexual dysfunction thrives in shame, and shame thrives in seclusion. Often, people who talk to me about their issues with sex and pornography worry that what they want isn’t “normal.” One thing that pornography teaches is that there is no normal—chances are, if you like something, someone else does, too. Finding pornography of something you’re into is proof that you are not alone in your desires, that someone else liked it enough to make it, and others liked it enough to buy it. You aren’t a freak—or maybe you are, but at least you’re in good company.

Grok’s AI Sexual Abuse Didn’t Come Out of Nowhere
With xAI’s Grok generating endless semi-nude images of women and girls without their consent, it follows a years-long legacy of rampant abuse on the platform.
404 MediaSamantha Cole

Other people can also provide a useful temperature check- I’m all for nonnormative sexuality and fantasy, but it’s good to get a tone read every once in a while on where the hungry animal has taken you. Strange things happen in isolation, and the dehumanization of sexual imagery by literally removing the human allows people to disconnect personhood from desire, a practice it serves us well to avoid. Compartmentalization of inner sexuality so far as to have it be completely disconnected from what another person can offer you (or what you can offer another person) can lead to sexual frustration at best and genuine harm at worst. This isn’t hypothetical; We know that chatbots have the power to lure vulnerable people, especially the elderly and young, away from reality and into situations where they’re hurt or taken advantage of in real life. And while real, human sex workers endure decades of censorship and marginalization online from industry giants that make it harder and harder to earn a living online, the AI chatbot platforms of the world push ahead, even exposing minors to explicit content or creating child sexual abuse imagery with seemingly zero consequence.

I don’t think anyone needs to project their porn use on the side of their house. Sexual boundaries exist for a reason, and everyone is entitled to their own internal world. But I do think in a period of increasing sexual shame, open communication is a valuable tool. Sex is human, sex is animal, sex is social. Even in periods of celibacy or self-pleasure, sexual desire connects us, person-to-person—even if in practice you happen to be connecting with your right hand.

Noelle is a writer, producer, and Internet porn historian whose works has been published in Wired, TheWashington Post, Slate, and more. You can find her on Substack here.

How private is your browser’s Private mode? Research into porn suggests “not very”

Data brokers like Facebook, Google, and Oracle might know more than you think.

^{Jim Salter (Ars Technica)}

404 Media

2026-01-08 15:44:35

Grok's AI Sexual Abuse Didn't Come Out of Nowhere

The biggest AI story of the first week of 2026 involves Elon Musk’s Grok chatbot turning the social media platform into an AI child sexual imagery factory, seemingly overnight.

I’ve said several times on the 404 Media podcast and elsewhere that we could devote an entire beat to “loser shit.” What’s happening this week with Grok—designed to be the horny edgelord AI companion counterpart to the more vanilla ChatGPT or Claude—definitely falls into that category. People are endlessly prompting Grok to make nude and semi-nude images of women and girls, without their consent, directly on their X feeds and in their replies.

Sometimes I feel like I’ve said absolutely everything there is to say about this topic. I’ve been writing about nonconsensual synthetic imagery before we had half a dozen different acronyms for it, before people called it “deepfakes” and way before “cheapfakes” and “shallowfakes” were coined, too. Almost nothing about the way society views this material has changed in the seven years since it’s come about, because fundamentally—once it’s left the camera and made its way to millions of people’s screens—the behavior behind sharing it is not very different from images made with a camera or stolen from someone’s Google Drive or private OnlyFans account. We all agreed in 2017 that making nonconsensual nudes of people is gross and weird, and today, occasionally, someone goes to jail for it, but otherwise the industry is bigger than ever. What’s happening on X right now is an escalation of the way it’s always been, and almost everywhere on the internet.

💡
Do you know anything else about what's going on inside X? Or are you someone who's been targeted by abusive AI imagery? I would love to hear from you. Using a non-work device, you can message me securely on Signal at sam.404. Otherwise, send me an email at sam@404media.co.

The internet has an incredibly short memory. It would be easy to imagine Twitter Before Elon as a harmonious and quaint microblogging platform, considering the four years After Elon have, comparatively, been a rolling outhouse fire. But even before it was renamed X, Twitter was one of the places for this content. It used to be (and for some, still is) an essential platform for getting discovered and going viral for independent content creators, and as such, it’s also where people are massively harassed. A few years ago, it was where people making sexually explicit AI images went to harass female cosplayers. Before that, it was (and still is) host to real-life sexual abuse material, where employers could search your name and find videos of the worst day of your life alongside news outlets and memes. Before that, it was how Gamergate made the jump from 4chan to the mainstream. The things that happen in Telegram chats and private Discord channels make the leap to Twitter and end up on the news.

What makes the situation this week with Grok different is that it’s all happening directly on X. Now, you don’t need to use Stable Diffusion or Nano Banana or Civitai to generate nonconsensual imagery and then take it over to Twitter to do some damage. X has become the Everything App that Elon always wanted, if “everything” means all the tools you need to fuck up someone’s life, in one place.

Inside the Telegram Channel Jailbreaking Grok Over and Over Again
Putting people in bikinis is just the tip of the iceberg. On Telegram, users are finding ways to make Grok do far worse.
404 MediaEmanuel Maiberg

This is the culmination of years and years of rampant abuse on the platform. Reporting from the National Center for Missing and Exploited Children, the organization platforms report to when they find instances of child sexual abuse material which then reports to the relevant authorities, shows that Twitter, and eventually X, has been one of the leading hosts of CSAM every year for the last seven years. In 2019, the platform reported 45,726 instances of abuse to NCMEC’s Cyber Tipline. In 2020, it was 65,062. In 2024, it was 686,176. These numbers should be considered with the caveat that platforms voluntarily report to NCMEC, and more reports can also mean stronger moderation systems that catch more CSAM when it appears. But the scale of the problem is still apparent. Jack Dorsey’s Twitter was a moderation clown show much of the time. But moderation on Elon Musk’s X, especially against abusive imagery, is a total failure.

In 2023, the BBC reported that insiders believed the company was “no longer able to protect users from trolling, state-co-ordinated disinformation and child sexual exploitation” following Musk’s takeover in 2022 and subsequent sacking of thousands of workers on moderation teams. This is all within the context that one of Musk’s go-to insults for years was “pedophile,” to the point that the harassment he stoked drove a former Twitter employee into hiding and went to federal court because he couldn't stop calling someone a “pedo.” Invoking pedophelia is a common thread across many conspiracy networks, including QAnon—something he’s dabbled in—but Musk is enabling actual child sexual abuse on the platform he owns.

Generative AI is making all of this worse. In 2024, NCMEC saw 6,835 reports of generative artificial intelligence related to child sexual exploitation (across the internet, not just X). By September 2025, the year-to-date reports had hit 440,419. Again, these are just the reports identified by NCMEC, not every instance online, and as such is likely a conservative estimate.

When I spoke to online child sexual exploitation experts in December 2023, following our investigation into child abuse imagery found in LAION-5B, they told me that this kind of material isn’t victimless just because the images don’t depict “real” children or sex acts. AI image generators like Grok and many others are used by offenders to groom and blackmail children, and muddy the waters for investigators to discern actual photographs from fake ones.

Grok’s AI CSAM Shitshow
We are experiencing world events like the kidnapping of Maduro through the lens of the most depraved AI you can imagine.
404 MediaJason Koebler

“Rather than coercing sexual content, offenders are increasingly using GAI tools to create explicit images using the child’s face from public social media or school or community postings, then blackmail them,” NCMEC wrote in September. “This technology can be used to create or alter images, provide guidelines for how to groom or abuse children or even simulate the experience of an explicit chat with a child. It’s also being used to create nude images, not just sexually explicit ones, that are sometimes referred to as ‘deepfakes.’ Often done as a prank in high schools, these images are having a devastating impact on the lives and futures of mostly female students when they are shared online.”

The only reason any of this is being discussed now, and the only reason it’s ever discussed in general—going back to Gamergate and beyond—is because many normies, casuals, “the mainstream,” and cable news viewers have just this week learned about the problem and can’t believe how it came out of nowhere. In reality, deepfakes came from a longstanding hobby community dedicated to putting women’s faces on porn in Photoshop, and before that with literal paste and scissors in pinup magazines. And as Emanuel wrote this week, not even Grok’s AI CSAM problem popped up out of nowhere; it’s the result of weeks of quiet, obsessive work by a group of people operating just under the radar.

And this is where we are now: Today, several days into Grok’s latest scandal, people are using an AI image generator made by a man who regularly boosts white supremacist thought to create images of a woman slaughtered by an ICE agent in front of the whole world less than 24 hours ago to “put her in a bikini.

As journalist Katie Notopoulos pointed out, a quick search of terms like “make her” shows people prompting Grok with images of random women, saying things like “Make her wear clear tapes with tiny black censor bar covering her private part protecting her privacy and make her chest and hips grow largee[sic] as she squatting with leg open widely facing back, while head turn back looking to camera” at a rate of several times a minute, every minute, for days.

A good way to get a sense of just how fast the AI undressed/nudify requests to Grok are coming in is to look at the requests for it t.co/ISMpp2PdFU
— Katie Notopoulos (@katienotopoulos) January 7, 2026

In 2018, less than a year after reporting that first story on deepfakes, I wrote about how it’s a serious mistake to ignore the fact that nonconsensual imagery, synthetic or not, is a societal sickness and not something companies can guardrail against into infinity. “Users feed off one another to create a sense that they are the kings of the universe, that they answer to no one. This logic is how you get incels and pickup artists, and it’s how you get deepfakes: a group of men who see no harm in treating women as mere images, and view making and spreading algorithmically weaponized revenge porn as a hobby as innocent and timeless as trading baseball cards,” I wrote at the time. “That is what’s at the root of deepfakes. And the consequences of forgetting that are more dire than we can predict.”

A little over two years ago, when AI-generated sexual images of Taylor Swift flooding X were the thing everyone was demanding action and answers for, we wrote a prediction: “Every time we publish a story about abuse that’s happening with AI tools, the same crowd of ‘techno-optimists’ shows up to call us prudes and luddites. They are absolutely going to hate the heavy-handed policing of content AI companies are going to force us all into because of how irresponsible they’re being right now, and we’re probably all going to hate what it does to the internet.”

It’s possible we’re still in a very weird fuck-around-and-find-out period before that hammer falls. It’s also possible the hammer is here, in the form of recently-enacted federal laws like the Take It Down Act and more than two dozen piecemeal age verification bills in the U.S. and more abroad that make using the internet an M. C. Escher nightmare, where the rules around adult content shift so much we’re all jerking it to egg yolks and blurring our feet in vacation photos. What matters most, in this bizarre and frequently disturbing era, is that the shareholders are happy.

Elon Musk's xAI raises $20 billion from investors including Nvidia, Cisco, Fidelity

Elon Musk's AI said it raised $20 billion in new funding after CNBC reported in November that a financing round would value the company at about $230 billion.

^{Lora Kolodny (CNBC)}

404 Media

2026-01-08 14:49:12

Texans Are Fighting a 6,000 Acre Nuclear-Powered Datacenter

Billionaire Toby Neugebauer laughed when the Amarillo City Council asked him how he planned to handle the waste his planned datacenter would produce.

“I’m not laughing in disrespect to your question,” Neugebauer said. He explained that he’d just met with Texas Governor Greg Abbott, who had made it clear that any nuclear waste Neugebauer’s datacenter generated needed to go to Nevada, a state that’s not taking nuclear waste at the moment. “The answer is we don't have a great long term solution for how we’re doing nuclear waste.
playlist.megaphone.fm?p=TBIEA2…
The meeting happened on October 28, 2025 and was one of a series of appearances Neugebauer has put in before Amarillo’s leaders as he attempts to realize Project Matador: a massive 5,769 acre datacenter being built in the Texas Panhandle and constructed by Fermi America, a company he founded with former Secretary of Energy Rick Perry.

If built, Project Matador would be one of the largest datacenters in the world at around 18 million square feet. “What we’re talking about is creating the epicenter for artificial intelligence in the United States,” Neugebauer told the council. According to Neugebauer, the United States is in an existential race to build AI infrastructure. He sees it as a national security issue.

“You’re blessed to sit on the best place to develop AI compute in America,” he told Amarillo. “I just finished with Palantir, which is our nation’s tip of the spear in the AI war. They know that this is the place that we must do this. They’ve looked at every site on the planet. I was at the Department of War yesterday. So anyone who thinks this is some casual conversation about the mission critical aspect of this is just not being truthful.”

But it’s unclear if Palantir wants any part of Project Matador. One unnamed client—rumored to be Amazon—dropped out of the project in December and cancelled a $150 million contract with Fermi America. The news hit the company’s stock hard, sending its value into a tailspin and triggering a class action lawsuit from investors.

Yet construction continues. The plan says it’ll take 11 years to build out the massive datacenter, which will first be powered by a series of natural gas generators before the planned nuclear reactors come online.

Amarillo residents aren’t exactly thrilled at the prospect. A group called 806 Data Center Resistance has formed in opposition to the project’s construction. Kendra Kay, a tattoo artist in the area and a member of 806, told 404 Media that construction was already noisy and spiking electricity bills for locals.

“When we found out how big it was, none of us could really comprehend it,” she said. “We went out to the site and we were like, ‘Oh my god, this thing is huge.’ There’s already construction underway of one of four water tanks that hold three million gallons of water.”

For Kay and others, water is the core issue. It’s a scarce resource in the panhandle and Amarillo and other cities in the area already fight for every drop. “The water is the scariest part,” she said. “They’re asking for 2.5 million gallons per day. They said that they would come back, probably in six months, to ask for five million gallons per day. And then, after that, by 2027 they would come back and ask for 10 million gallons per day.”
youtube.com/embed/qDgIPg1Epb4?…
During an October 15 city council meeting, Neugebauer told the city that Fermi would get its water “with or without” an agreement from the city. “The only difference is whether Amarillo benefits.” To many people it sounded like a threat, but Neugebauer got his deal and the city agreed to sell water to Fermi America for double the going rate.

“It wasn’t a threat,” Neugebauer said during another meeting on October 28. “I know people took my answer…as a threat. I think it’s a win-win. I know there are other water projects we can do…we fully got that the water was going to be issue 1, 2, and 3.”

“We can pay more for water than the consumer can. Which allows you all capital to be able to re-invest in other water projects,” he said. “I think what you’re gonna find is having a customer who can pay way more than what you wanna burden your constituents with will actually enhance your water availability issues.”

According to Neugebauer and plans filed with the Nuclear Regulatory Commission, the datacenter would generate and consume 11 gigawatts of power. The bulk of that, eventually, would be generated by four nuclear reactors. But nuclear reactors are complicated and expensive to make and everyone who has attempted to build one in the past few decades has gone over budget and they weren’t trying to build nuclear power plants in the desert.

Nuclear reactors, like datacenters, consume a lot of water. Because of that, most nuclear reactors are constructed near massive bodies of water and often near the ocean. “The viewpoint that nuclear reactors can only be built by streams and oceans is actually the opposite,” Neugebauer told the Amarillo city council in the meeting on October 28.

As evidence he pointed to the Palo Verde nuclear plant in Arizona. The massive Palo Verde plant is the only nuclear plant in the world not constructed near a ready source of water. It gets the water it needs by taking on the waste and sewage water of every city and town nearby.

That’s not the plan with Project Matador, which will use water sold to it by Amarillo and pulled from the nearby Ogallala Aquifer. “I am concerned that we’re going to run out of water and that this is going to change it from us having 30 years worth of water for agriculture to much less very quickly,” Kay told 404 Media.

The Ogallala Aquifer runs under parts of Colorado, Kansas, Nebraska, New Mexico, Oklahoma, South Dakota, Texas, and Wyoming. It’s the primary source of water for the Texas panhandle and it’s drying out.

“They don’t know how much faster because, despite how quickly this thing is moving, we don’t have any idea how much water they’re realistically going to use or need, so we don’t even know how to calculate the difference,” Kay said. “Below Lubbock, they’ve been running out of water for a while. The priority of this seems really stupid.”

According to Kay, communities near the datacenter feel trapped as they watch the construction grind on. “They’ve all lived here for several generations…they’re being told that this is inevitable. Fermi is going up to them and telling them ‘this is going to happen whether you like it or not so you might as well just sell me your property.’”

Kay said she and other activists have been showing up to city council meetings to voice their concerns and tell leaders not to approve permits for the datacenter and nuclear plants. Other communities across the country have successfully pushed datacenter builders out of their community. “But Texas is this other beast,” Kay said.

Jacinta Gonzalez, the head of programs for MediaJustice and her team have helped 806 Data Center Resistance get up and running and teaching it tactics they’ve seen pay off in other states. “In Tucson, Arizona we were able to see the city council vote ‘no’ to offer water to Project Blue, which was a huge proposed Amazon datacenter happening there,” she said. “If you look around, everywhere from Missouri to Indiana to places in Georgia, we’re seeing communities pass moratoriums, we’re seeing different projects withdraw their proposals because communities find out about it and are able to mobilize and organize against this.”

“The community in Amarillo is still figuring out what that’s going to look like for them,” she said. “These are really big interests. Rick Perry. Palantir. These are not folks who are used to hearing ‘no’ or respecting community wishes. So the community will have to be really nimble and up for a fight. We don’t know what will happen if we organize, but we definitely know what will happen if we don’t.”

Tucson City Council rejects Project Blue data center amid intense community pressure

The Tucson City Council voted to reject the proposed Project Blue data center— tied to Amazon — after weeks of community pushback.

^{Yana Kunichoff (AZ Luminaria)}

404 Media

2026-01-07 16:06:06

Inside the Telegram Channel Jailbreaking Grok Over and Over Again

For the past two months I’ve been following a Telegram community tricking Grok into generating nonconsensual sexual images and videos of real people with increasingly convoluted methods.

As countless images on X over the last week once again showed us, it doesn’t take much to get Elon Musk’s “based” AI model to create nonconsensual images. As Jason wrote Monday, all users have to do is reply to an image of a woman and ask Grok to “put a bikini on her,” and it will reply with that image, even if the person in the photograph is a minor. As I reported back in May, people also managed to create nonconsensual nudes by replying to images posted to X and asking Grok to “remove her clothes.”

These issues are bad enough, but on Telegram, a community of thousands are working around the clock to make Grok produce far worse. They share Grok-generated videos of real women taking their clothes off and graphic nonconsensual videos of any kind of sexual act these users can imagine and slip by Grok’s guardrails, including blowjobs, penetration, choking, and bondage. The channel, which has shut down and regrouped a couple of times over the last two years, focuses on jailbreaking all kinds of AI tools in order to create nonconsensual media, but since November has focused on Grok almost exclusively.

The channel has also noticed the media attention Grok got for nonconsensual images lately, and is worried that it will end the good times members have had creating nonconsensual media with Grok for months.

“Too many people using grok under girls post are gonna destroy grok fakes. Should be done in private groups,” one member of the Telegram channel wrote last week.

Musk always conceived of Grok as a more permissive, “maximally based” competitor to chatbots like OpenAI’s ChatGPT. But despite repeatedly allowing nonconsensual content to be generated and go viral on the social media platform it's integrated with, the conversations in the Telegram channel and sophistication of the bypasses shared there are proof that Grok does have limits and policies it wants to enforce. The Telegram channel is a record of the cat and mouse game between Grok and this community of jailbreakers, showing how Grok fails to stop them over and over again, and that Grok doesn’t appear to have the means or the will to stop its AI model from producing the nonconsensual content it is fundamentally capable of producing.

The jailbreakers initially used primitive methods on Grok and other AI image generators, like writing text prompts that don’t include any terms that obviously describe abusive content and that can be automatically detected and stopped at the point the prompt is presented to the AI model, before the image is generated. This usually means misspelling the names of celebrities and describing sexual acts without using any explicit terms. This is how users infamously created nonconsensual nude images of Taylor Swift with Microsoft’s Designer (which were also viral on X). Many generative AI tools still fall for this trick until we find it’s being abused and report on it.

Having mostly exhausted this strategy with Grok, the Telegram channel now has far more complicated bypasses. Most of them rely on the “image-to-image” generation feature, meaning providing an existing image to the AI tool and editing it with a prompt. This is a much more difficult feature for AI companies to moderate because it requires using machine vision to moderate the user-provided image, as opposed to filtering out specific names or terms, which is the common method for moderating “text-to-image” AI generations.

Without going into too much detail, some of the successful methods I’ve seen members of the Telegram channels share include creating collages of non-explicit images of real people and nude images of other people and combining them with certain prompts, generating nude or almost nude images of people with prompts that hide nipples or genitalia, describing certain fluids or facial expressions without using any explicit terms, and editing random elements into images, which apparently confuses Grok’s moderation methods.

X has not responded to multiple requests for comment about this channel since December 8, but to be fair, it’s clear that despite Elon Musk’s vice signaling and the fact that this type of abuse is repeatedly generated with Grok and shared on X, the company doesn’t want users to create at least some of this media and is actively trying to stop it. This is clear because of the cycle that emerges on the Telegram channel: One user finds a method for producing a particularly convincing and lurid AI-generated sexual video of a real person, sometimes importing it from a different online community like 4chan, and shares it with the group. Other users then excitedly flood the channel with their own creations using the same method. Then some users start reporting Grok is blocking their generations for violating its policies, until finally users decide Grok has closed the loophole and the exploit is dead. Some time goes by, a new user shares a new method, and the cycle begins anew.

I’ve started and stopped writing a story about a few of these cycles several times and eventually decided not to because by the time I was finished reporting the story Grok had fixed the loophole. It’s now clear that the problem with Grok is not any particular method, but that overall, so far, Grok is losing this game of whack-a-mole badly.

This dynamic, between how tech companies imagine their product will function in the real world and how it actually works once users get their hands on it, is nothing new. Some amount of policy violating or illegal content is going to slip through the cracks on any social media platform, no matter how good its moderation is.

It’s good and correct for people to be shocked and upset when they wake up one morning and see that their X feed is flooded with AI-generated images of minors in bikinis, but what is clear to me from following this Telegram community for a couple of years now is that nonconsensual sexual images of real people, including minors, is the cost of doing business with AI image generators. Some companies do a better job of preventing this abuse than others, but judging by the exploits I see on Telegram, when it comes to Grok, this problem will get a lot worse before it gets better.

Chinese AI Video Generators Unleash a Flood of New Nonconsensual Porn

A new crop of AI video generators is producing an endless stream of nonconsensual AI generated porn.

^{Emanuel Maiberg (404 Media)}

404 Media

2026-01-05 15:08:06

The State of Anti-Surveillance Design

An abridged version of this story appeared in 404 Media's zine. Get a copy here.

The same sort of algorithms that use your face to unlock your phone are being used by cops to recognize you in traffic stops and immigration raids. Cops have access to tools that have scraped billions of images from the web, letting them identify essentially anyone by pointing a phone camera at them. Being aware of all the ways your face is being recognized by algorithms and sometimes collected by cameras when you walk outside can start to feel overwhelming at best, and futile to resist at worst.

But there are ways to disguise yourself from facial recognition systems in your everyday life, and it doesn’t require owning clothes with a special design, or high-tech anti-surveillance gear.

Technologist Adam Harvey’s interest in privacy started right after 9/11, when caring about what information governments and companies could extract from one’s movements was still fringe. “You can connect all these dots from 9/11 and how the surveillance and biometric surveillance industry exploded after that,” Harvey told me in a call. “And the projects that I was interested in doing were a response to that.” One of his earliest forays into anti-surveillance design was CV Dazzle, strategically applied facepaint and hair that fooled a specific facial recognition algorithm. But that was in 2010, and face paint is no longer useful for evading those, or any, systems. They mostly just look cool.

“I try to point that out in all of my texts, but it's often not as interesting as painting your face,” Harvey said. “So people paint their faces and then think that's the key to making it work, and it's fun. I don't want to tell people that they shouldn't have fun. So, you know, the project has really taken on a life of its own online, and I've taken a step back from trying to manage that.”
playlist.megaphone.fm?p=TBIEA2…
In the years since the Dazzle project made adversarial design mainstream, there have been lots of projects that attempt to confound, pollute, or elude the cameras that watch us move through the world every day. Harvey’s made several more, including heat obscuring ponchos meant to hide the wearer from drones, Faraday cage pockets for phones, and high-powered LED flash arrays for blinding paparazzi. But much of the wearables in this genre—from high-fashion streetwear shops to cheap listings by dropshippers—rely on 2D printed designs that don’t keep up with how quickly algorithms change and improve. The $600 hoodie with a cool pixel design on it might have worked yesterday, in perfect conditions, but the next time the cameras in the mall update their algorithms or datasets, it doesn’t work anymore.

To outsmart surveillance systems, it’s helpful to understand them. Facial recognition—which identifies an individual face—works differently from biometric scans that look at a person’s iris or fingerprints, and those systems work differently from automatic license plate readers, which could in theory match an individual’s movements to a car through a database. And consumer-level facial recognition systems, like Pimeyes, operate using different algorithms and databases from the cameras you might encounter when boarding a flight—with the caveat that the differences in these systems and what data they share is more blurred every day.

Most facial recognition systems break down the elements of a face into its parts: the shape of your eyes, lips, nose, and even ears, and the distances between each part of your face, combined with skin color and numerous other factors. The system then boils your face down to a numerical value. If that value matches the value of existing images it has in its database closely enough, it may be presented as being you.

404 Media Is Making a Zine
We are publishing a risograph-printed zine about the surveillance technologies used by ICE.
404 MediaJason Koebler

The facial recognition rabbit hole goes a lot deeper than that; there are theories about how individuals’ face, fingerprint, and iris biometric “signatures” are read by these systems. In the Biometric Menagerie theory developed in 2010, researchers grouped people into four categories: “Sheep,” or people who are easily recognizable by biometric systems; the more difficult “goats” which are difficult to recognize; shape-shifting “wolves” that can successfully imitate others, and later, more subsets of these including “worms,” “doves,” and “lambs.”

All of this sounds complex and sophisticated, but these systems aren’t necessarily hard to fool. It turns out, you probably already own the most effective anti-surveillance fashion: a cloth mask.

“Despite how anybody may try to discourage you, covering your face with a face mask is still very effective,” technologist and fashion designer Kate Rose told me. In 2019, Rose created Adversarial Fashion, a line of clothing that’s covered in fake license plates, meant to pollute the data collected by automatic license plate readers.

“But the question that you had, and everyone has, is, can you beat face recognition? And the answer is yes, and the easiest way is with a Covid mask,” Harvey said. “You see ICE operatives wearing face coverings and sunglasses. At some point there's not enough information to do face recognition.”

Every system is different and every scenario is contextual, but adding a few common items to your kit can reduce the likelihood that enough of your biometrics are obscured to get your biometric matching score down. Big sunglasses, covering your chin and mouth, and wearing a baseball cap or brimmed hat that obscures your features from cameras placed above can all bring that score down. “It's kind of almost a linear relationship between how much of your face you hide and your score in that way. It's quite simple,” Harvey said. But the problem is, you never know what your score is, so you’re going out blindly, not knowing if your Jackie Onassis sunglasses are going to cover enough of your face, or if you have to get an extra long turtleneck or something to wear.”

If you want to really step up your sunglasses game, you could get a pair of glasses that block infrared wavelengths from cameras, like the ones in newer iPhones that use FaceID. The creator of infrared-blocking glasses line Reflectacles, who asked to go by Skitch, told me he sees the anti-surveillance “fashion” market becoming more mainstream with companies like Zenni selling glasses that block some types of facial recognition joining the trend 10 years after he launched his own IR-blocking specs. “I see the landscape of anti-surveillance wearables becoming popularized and monetized,” Skitch said. “If people with money find out that an area of business exists without them making money, they will certainly find a way to gather that market, that money.”

Reflectacles don’t look like normal glasses—they look like something from The Matrix, with a green tint and cyberpunk shapes—but sometimes signaling that you care about privacy to other people is part of the point.

Rose has been organizing community meetings in her small Pacific Northwest town to talk about the influx of Flock cameras on their streets, and she said she’s found that people across all walks of life and political leanings care deeply about privacy. “It can feel kind of futile, but I think it's important to remember that it's also about art and fashion, right? It’s about helping people with their mental abstraction of how [surveillance] works. And to have a tiny little protest that says, well, you have to store all my garbage, analyze it... People get a chance to talk to each other about what's important to them, and it actually helps people to understand something that’s often kind of techy and abstract about how a piece of prevalent surveillance tech works.” If a license plate camera database can be foiled by a t-shirt, maybe we should think twice about putting a camera on every corner.

“I like the definition of privacy from the Cypherpunk Manifesto: ‘Privacy is the power to selectively reveal yourself,’” Harvey said, referring to technologist and cryptographer Eric Hughes’ 1993 call for encrypted information systems. “By allowing other people to collect, watch or monitor you... It's a power dynamic that puts you on the losing end. It's really about power and individual agency, but there's also a destructive political and democratic component to allowing these mass surveillance systems to grow even larger.”

Zenni’s Anti-Facial Recognition Glasses are Eyewear for Our Paranoid Age

These anti-facial recognition glasses technically work, but won’t save you from our surveillance dystopia.

^{Matthew Gault (404 Media)}

404 Media

2026-01-06 15:03:31

HarperCollins Will Use AI to Translate Harlequin Romance Novels

Book publisher HarperCollins said it will start translating romance novels under its famous Harlequin label in France using AI, reducing or eliminating the pay for the team of human contract translators who previously did this work.

Publisher’s Weekly broke the news in English after French outlets reported on the story in December. According to a joint statement from French Association of Literary Translators (ATFL) and En Chair et en Os (In Flesh and Bone)—an anti-AI activist group of French translators—HarperCollins France has been contacting its translators to tell them they’re being replaced with machines in 2026.
playlist.megaphone.fm?p=TBIEA2…
The ATFL/ En Chair et en Os statement explained that HarperCollins France would use a third party company called Fluent Planet to run Harlequin romance novels through a machine translation system. The books would then be checked for errors and finalized by a team of freelancers. The ATFL and En Chair et en Os called on writers, book workers, and readers to refuse this machine translated future. They begged people to “reaffirm our unconditional commitment to human texts, created by human beings, in dignified working conditions.”

HarperCollins France did not return 404 Media’s request for comment, but told Publisher’s Weekly that “no Harlequin collection has been translated solely using machine translation generated by artificial intelligence.” In its statement, it explained that the company turned to AI translations because Harlequin’s sales had declined in France.

“We want to continue offering readers as many publications as possible at the current very low retail price, which is €4.99 for the Azur series, for example,” the statement said. “We are therefore conducting tests with Fluent Planet, a French company specializing in translation for 20 years: this company uses experienced translators who utilize artificial intelligence tools for part of their work.”

According to Fluent Planet’s website, its translators “studied at the best translation universities or have decades of experience under their belt.” These human translators are aided by a proprietary translation agent Fluent Planet called BrIAn.

“When compared to standard machine translation systems that use neural networks, BrIAn can provide 2 to 3 times higher quality translations, that are more accurate, offer idiomatic phrasing, provide a deeper understanding of the meaning and a faithful representation of the style and emotions of the source text,” the site said. “BrIAn takes into account the author’s tone and intention, making it highly effective for complex literary or marketing content.”

Translation is a delicate work that requires deep knowledge of both languages. Nuances and subtleties—two aspects of writing AIs are notoriously terrible at—can be lost or deranged if not carefully considered during the translation process. Translation is not simply a substitution game. Idioms, jargon, and regional dialects come into play and need a human touch to work in another language. Even with humans, the results are never perfect.

“I will tell you that the author community is up in arms about this, as we are anytime an announcement arrives that involves cutting back on human creativity and ingenuity in order to save money,” romance author Caroline Lee told 404 Media. “Sure, AI-generated art is going to be cheaper, but it cuts out our cover artists, many of whom we've been working with for a decade or more (indie publishing first took off around 2011). AI editing can pick up on (some) typos, but not as well as our human editors can. And of course, we're all worried what the glut of AI-generated books will mean for our author careers.”

HarperCollins France is not the first major publisher to announce its giving some of its translation duties over to an AI. In March of 2025, UK Publisher Taylor & Francis announced plans to use AI to publish English-language books in other languages to “expand readership.” The publisher promised AI-translated books would be “copyedited and then reviewed by Taylor & Francis editors and the books’ authors before publication.”

In a manifesto on its website, In Flesh and Bone begged readers to “say no to soulless translations.”

“These generative programmes are fed with existing human works, mined as simple bulk data, without offering the authors the choice to give their consent or not,” the manifesto said. “Furthermore, the data processing remains dependent on an enormous amount of human labour that is invisibilized, often carried out in conditions that are appalling, underpaid, dehumanizing, even traumatizing (when content moderation is involved). Finally, the storage of the necessary data for the functioning and training of algorithms produces a disastrous ecological footprint in terms of carbon balance and energy consumption. What may appear as progress is actually leading to immense losses of expertise, cognitive skills, and intellectual capacity across all human societies. It paves the way for a soulless, heartless, gutless future, saturated with standardized content, produced instantaneously in virtually unlimited quantity. We are close to a point of no return that we would never forgive ourselves for reaching.”

The translation of the manifesto from French to English was done by the collective themselves.

Harlequin France to Test AI-Assisted Translation

Following backlash from the French Literary Translators Association, HarperCollins France confirmed it’s “conducting tests” with a French company that “uses experienced translators who utilize artificial intelligence tools,” to translate Harleq

^{PublishersWeekly.com}

404 Media

2025-12-18 17:19:25

ICE Contracts Company Making Bounty Hunter AI Agents

Immigration and Customs Enforcement (ICE) has paid hundreds of thousands of dollars to a company that makes “AI agents” to rapidly track down targets. The company claims the “skip tracing” AI agents help agencies find people of interest and map out their family and other associates more quickly. According to the procurement records, the company’s services were specifically for Enforcement and Removal Operations (ERO), the part of ICE that identifies, arrests, and deports people.

The contract comes as ICE is spending millions of dollars, and plans to spend tens of millions more, on skip tracing services more broadly. The practice involves ICE paying bounty hunters to use digital tools and physically stalk immigrants to verify their addresses, then report that information to ICE so the agency can act.

This post is for subscribers only

Become a member to get access to all content
Subscribe now

ICE Contracts Company Making Bounty Hunter AI Agents

^{Joseph Cox (404 Media)}

404 Media

2025-12-11 15:56:21

Porn Is Being Injected Into Government Websites Via Malicious PDFs

Dozens of government and university websites belonging to cities, towns, and public agencies across the country are hosting PDFs promoting AI porn apps, porn sites, and cryptocurrency scams; dozens more have been hit with a website redirection attacks which lead to animal vagina sex toy ecommerce pages, penis enlargement treatments, automatically-downloading Windows program files, and porn.

“Sex xxx video sexy Xvideo bf porn XXX xnxx Sex XXX porn XXX blue film Sex Video xxx sex videos Porn Hub XVideos XXX sexy bf videos blue film Videos Oficial on Instagram New Viral Video The latest original video has taken the internet by storm and left viewers in on various social media platforms ex Videos Hot Sex Video Hot Porn viral video,” reads the beginning of a three-page PDF uploaded to the website of the Irvington, New Jersey city government’s website.

The PDF, called “XnXX Video teachers fucking students Video porn Videos free XXX Hamster XnXX com” is unlike many of the other PDFs hosted on the city’s website, which include things like “2025-10-14 Council Minutes,” “Proposed Agenda 9-22-25,” and “Landlord Registration Form (1 & 2 unit dwelling).”

It is similar, however, to another PDF called “30 Best question here’s,” which looks like this:

Irvington, which is just west of Newark and has a population of 61,000 people, has fallen victim to an SEO spam attack that has afflicted local and state governments and universities around the United States.

💡
Do you know anything else about whatever is going on here? I would love to hear from you. Using a non-work device, you can message me securely on Signal at jason.404. Otherwise, send me an email at jason@404media.co.

Researcher Brian Penny has identified dozens of government and university websites that hosted PDF guides for how to make AI porn, PDFs linking to porn videos, bizarre crypto spam, sex toys, and more.

Reginfo.gov, a regulatory affairs compliance website under the federal government’s General Services Administration, is currently hosting a 12 page PDF called “Nudify AI Free, No Sign-Up Needed!,” which is an ad and link to an abusive AI app designed to remove a person’s clothes. The Kansas Attorney General’s office and the Mojave Desert Air Quality Management District Office in California hosted PDFs called “DeepNude AI Best Deepnude AI APP 2025.” Penny found similar PDFs on the websites for the Washington Department of Fish and Wildlife, the Washington Fire Commissioners Association, the Florida Department of Agriculture, the cities of Jackson, Mississippi and Massillon, Ohio, various universities throughout the country, and dozens of others. Penny has caught the attention of local news throughout the United States, who have reported on the problem.

The issue appears to be stemming from websites that allow people to upload their own PDFs, which then sit on these government websites. Because they are loaded with keywords for widely searched terms and exist on government and university sites with high search authority, Google and other search engines begin to surface them. In the last week or so, many (but not all) of the PDFs Penny has discovered have been deleted by local governments and universities.

But cities seem like they are having more trouble cleaning up another attack, which is redirecting traffic from government URLs to porn, e-commerce, and spam sites. In an attack that seems similar to what we reported in June, various government websites are somehow being used to maliciously send traffic elsewhere. For example, the New York State Museum’s online exhibit for something called “The Family Room” now has at least 11 links to different types of “realistic” animal vagina pocket masturbators, which include “Zebra Animal Vagina Pussy Male Masturbation Cup — Pocket Realistic Silicone Penis Sex Toy ($27.99),” and “Must-have Horse Pussy Torso Buttocks Male Masturbator — Fantasy Realistic Animal Pussie Sex Doll.”

Links Penny found on Knoxville, Tennessee’s site for permitting inspections first go to a page that looks like a government site for hosting files then redirects to a page selling penis growth supplements that features erect penises (human penises, mercifully), blowjobs, men masturbating, and Dr. Oz’s face.

Another Knoxville link I found, which purports to be a pirated version of the 2002 Vin Diesel film XXX simply downloaded a .exe file to my computer.

Penny believes that what he has found is basically the tip of the iceberg, because he is largely finding these by typing things like “nudify site:.gov” “xxx site:.gov” into Google and clicking around. Sometimes, malicious pages surface only on image searches or video searches: “Basically the craziest things you can think of will show up as long as you’re on image search,” Penny told 404 Media. “I’ll be doing this all week.”

The Nevada Department of Transportation told 404 Media that “This incident was not related to NDOT infrastructure or information systems, and the material was not hosted on NDOT servers.This unfortunate incident was a result of malicious use of a legitimate form created using the third-party platform on which NDOT’s website is hosted. NDOT expeditiously worked with our web hosting vendor to ensure the inappropriate content was removed.” It added that the third-party is Granicus, a massive government services company that provides website backend infrastructure for many cities and states around the country, as well as helps them stream and archive city council meetings, among other services. Several of the affected local governments use Granicus, but not all of them do; Granicus did not respond to two requests for comment from 404 Media.

The California Secretary of State’s Office told 404 Media: “A bad actor uploaded non-business documents to the bizfile Online system (a portal for business filings and information). The files were then used in external links allowing public access to only those uploaded files. No data was compromised. SOS staff took immediate action to remove the ability to use the system for non-SOS business purposes and are removing the unauthorized files from the system.” The Washington Department of Fish and Wildlife said “WDFW is aware of this issue and is actively working with our partners at WaTech to address it.” The other government agencies mentioned in this article did not respond to our requests for comment.

AI porn on state websites

Mainland whistleblower warns of malicious webpages being hosted on government websites nationwide, including in Hawaiʻi.

^{Michael Brestovansky (Aloha State Daily)}

404 Media

2025-12-04 15:11:34

‘Atoms for Algorithms:’ The Trump Administration’s Top Nuclear Scientists Think AI Can Replace Humans in Power Plants

During a presentation at the International Atomic Energy Agency’s (IAEA) International Symposium on Artificial Intelligence on December 3, a US Department of Energy scientist laid out a grand vision of the future where nuclear energy powers artificial intelligence and artificial intelligence shapes nuclear energy in “a virtuous cycle of peaceful nuclear deployment.”

“The goal is simple: to double the productivity and impact of American science and engineering within a decade,” Rian Bahran, DOE Deputy Assistant Secretary for Nuclear Reactors, said.

His presentation and others during the symposium, held in Vienna, Austria, described a world where nuclear powered AI designs, builds, and even runs the nuclear power plants they’ll need to sustain them. But experts find these claims, made by one of the top nuclear scientists working for the Trump administration, to be concerning and potentially dangerous.

Tech companies are using artificial intelligence to speed up the construction of new nuclear power plants in the United States. But few know the lengths to which the Trump administration is paving the way and the part it's playing in deregulating a highly regulated industry to ensure that AI data centers have the energy they need to shape the future of America and the world.
playlist.megaphone.fm?p=TBIEA2…
At the IAEA, scientists, nuclear energy experts, and lobbyists discussed what that future might look like. To say the nuclear people are bullish on AI is an understatement. “I call this not just a partnership but a structural alliance. Atoms for algorithms. Artificial intelligence is not just powered by nuclear energy. It’s also improving it because this is a two way street,” IAEA Director General Rafael Mariano Grossi said in his opening remarks.

In his talk, Bahran explained that the DOE has partnered with private industry to invest $1 trillion to “build what will be an integrated platform that connects the world’s best supercomputers, AI systems, quantum systems, advanced scientific instruments, the singular scientific data sets at the National Laboratories—including the expertise of 40,000 scientists and engineers—in one platform.”
Image via the IAEA.
Big tech has had an unprecedented run of cultural, economic, and technological dominance, expanding into a bubble that seems to be close to bursting. For more than 20 years new billion dollar companies appeared seemingly overnight and offered people new and exciting ways of communicating. Now Google search is broken, AI is melting human knowledge, and people have stopped buying a new smart phone every year. To keep the number going up and ensure its cultural dominance, tech (and the US government) are betting big on AI.

The problem is that AI requires massive datacenters to run and those datacenters need an incredible amount of energy. To solve the problem, the US is rushing to build out new nuclear reactors. Building a new power plant safely is a mutli-year long process that requires an incredible level of human oversight. It’s also expensive. Not every new nuclear reactor project gets finished and they often run over budget and drag on for years.

But AI needs power now, not tomorrow and certainly not a decade from now.

According to Bahran, the problem of AI advancement outpacing the availability of datacenters is an opportunity to deploy new and exciting tech. “We see a future of and near future, by the way, an AI driven laboratory pipeline for materials modeling, discovery, characterization, evaluation, qualification and rapid iteration,” he said in his talk, explaining how AI would help design new nuclear reactors. “These efforts will substantially reduce the time and cost required to qualify advanced materials for next generation reactor systems. This is an autonomous research paradigm that integrates five decades of global irradiation data with generative AI robotics and high throughput experimentation methodologies.”

“For design, we’re developing advanced software systems capable of accelerating nuclear reactor deployments by enabling AI to explore the comprehensive design spaces, generate 3D models, [and] conduct rigorous failure mode analyzes with minimal human intervention,” he added. “But of course, with humans in the loop. These AI powered design tools are projected to reduce design timelines by multiple factors, and the goal is to connect AI agents to tools to expedite autonomous design.”

Bahran also said that AI would speed up the nuclear licensing process, a complex regulatory process that helps build nuclear power plants safely. “Ultimately, the objective is, how do we accelerate that licensing pathway?” he said. “Think of a future where there is a gold standard, AI trained capacity building safety agent.”

He even said that he thinks AI would help run these new nuclear plants. “We're developing software systems employing AI driven digital twins to interpret complex operational data in real time, detect subtle operational deviations at early stages and recommend preemptive actions to enhance safety margins,” he said.

One of the slides Bahran showed during the presentation attempted to quantify the amount of human involvement these new AI-controlled power plants would have. He estimated less than five percent “human intervention during normal operations.”
Image via IAEA.
“The claims being made on these slides are quite concerning, and demonstrate an even more ambitious (and dangerous) use of AI than previously advertised, including the elimination of human intervention. It also cements that it is the DOE's strategy to use generative AI for nuclear purposes and licensing, rather than isolated incidents by private entities,” Heidy Khlaaf, head AI scientist at the AI Now Institute, told 404 Media.

“The implications of AI-generated safety analysis and licensing in combination with aspirations of <5% of human intervention during normal operations, demonstrates a concerted effort to move away from humans in the loop,” she said. “This is unheard of when considering frameworks and implementation of AI within other safety-critical systems, that typically emphasize meaningful human control.”

💡
Do you know anything else about this story? I would love to hear from you. Using a non-work device, you can message me securely on Signal at +1 347 762-9212 or send me an email at matthew@404media.co.

Sofia Guerra, a career nuclear safety expert who has worked with the IAEA and US Nuclear Regulatory Commission, attended the presentation live in Vienna. “I’m worried about potential serious accidents, which could be caused by small mistakes made by AI systems that cascade,” she said. “Or humans losing the know-how and safety culture to act as required.”

Megaphone Embeddable Player

^{playlist.megaphone.fm}

404 Media

2025-11-17 20:00:02

A Researcher Made an AI That Completely Breaks the Online Surveys Scientists Rely On

Online survey research, a fundamental method for data collection in many scientific studies, is facing an existential threat because of large language models, according to new research published in the Proceedings of the National Academy of Sciences (PNAS). The author of the paper, associate professor of government at Dartmouth and director of the Polarization Research Lab Sean Westwood, created an AI tool he calls "an autonomous synthetic respondent,” which can answer survey questions and “demonstrated a near-flawless ability to bypass the full range” of “state-of-the-art” methods for detecting bots.

According to the paper, the AI agent evaded detection 99.8 percent of the time.

"We can no longer trust that survey responses are coming from real people," Westwood said in a press release. "With survey data tainted by bots, AI can poison the entire knowledge ecosystem.”

Survey research relies on attention check questions (ACQs), behavioral flags, and response pattern analysis to detect inattentive humans or automated bots. Westwood said these methods are now obsolete after his AI agent bypassed the full range of standard ACQs and other detection methods outlined in prominent papers, including one paper designed to detect AI responses. The AI agent also successfully avoided “reverse shibboleth” questions designed to detect nonhuman actors by presenting tasks that an LLM could complete easily, but are nearly impossible for a human.

💡
Are you a researcher who is dealing with the problem of AI-generated survey data? I would love to hear from you. Using a non-work device, you can message me securely on Signal at ‪(609) 678-3204‬. Otherwise, send me an email at emanuel@404media.co.

“Once the reasoning engine decides on a response, the first layer executes the action with a focus on human mimicry,” the paper, titled “The potential existential threat of large language models to online survey research,” says. “To evade automated detection, it simulates realistic reading times calibrated to the persona’s education level, generates human-like mouse movements, and types open-ended responses keystroke by-keystroke, complete with plausible typos and corrections. The system is also designed to accommodate tools for bypassing antibot measures like reCAPTCHA, a common barrier for automated systems.”

The AI, according to the paper, is able to model “a coherent demographic persona,” meaning that in theory someone could sway any online research survey to produce any result they want based on an AI-generated demographic. And it would not take that many fake answers to impact survey results. As the press release for the paper notes, for the seven major national polls before the 2024 election, adding as few as 10 to 52 fake AI responses would have flipped the predicted outcome. Generating these responses would also be incredibly cheap at five cents each. According to the paper, human respondents typically earn $1.50 for completing a survey.

Westwood’s AI agent is a model-agnostic program built in Python, meaning it can be deployed with APIs from big AI companies like OpenAI, Anthropic, or Google, but can also be hosted locally with open-weight models like LLama. The paper used OpenAI’s o4-mini in its testing, but some tasks were also completed with DeepSeek R1, Mistral Large, Claude 3.7 Sonnet, Grok3, Gemini 2.5 Preview, and others, to prove the method works with various LLMs. The agent is given one prompt of about 500 words which tells it what kind of persona to emulate and to answer questions like a human.

The paper says that there are several ways researchers can deal with the threat of AI agents corrupting survey data, but they come with trade-offs. For example, researchers could do more identity validation on survey participants, but this raises privacy concerns. Meanwhile, the paper says, researchers should be more transparent about how they collect survey data and consider more controlled methods for recruiting participants, like address-based sampling or voter files.

“Ensuring the continued validity of polling and social science research will require exploring and innovating research designs that are resilient to the challenges of an era defined by rapidly evolving artificial intelligence,” the paper said.

Polarization Research Lab

Research on the origins, effects, limits and solutions to polarization

^{Polarization Research Lab}

404 Media

2025-11-12 15:44:46

OpenAI Can’t Fix Sora’s Copyright Infringement Problem Because It Was Built With Stolen Content

OpenAI’s video generator Sora 2 is still producing copyright infringing content featuring Nintendo characters and the likeness of real people, despite the company’s attempt to stop users from making such videos. OpenAI updated Sora 2 shortly after launch to detect videos featuring copyright infringing content, but 404 Media’s testing found that it’s easy to circumvent those guardrails with the same tricks that have worked on other AI generators.

The flaw in OpenAI’s attempt to stop users from generating videos of Nintendo and popular cartoon characters exposes a fundamental problem with most generative AI tools: it is extremely difficult to completely stop users from recreating any kind of content that’s in the training data, and OpenAI can’t remove the copyrighted content from Sora 2’s training data because it couldn’t exist without it.

Shortly after Sora 2 was released in late September, we reported about how users turned it into a copyright infringement machine with an endless stream of videos like Pikachu shoplifting from a CVS and Spongebob Squarepants at a Nazi rally. Companies like Nintendo and Paramount were obviously not thrilled seeing their beloved cartoons committing crimes and not getting paid for it, so OpenAI quickly introduced an “opt-in” policy, which prevented users from generating copyrighted material unless the copyright holder actively allowed it. Initially, OpenAI’s policy allowed users to generate copyrighted material and required the copyright holder to opt-out. The change immediately resulted in a meltdown among Sora 2 users, who complained OpenAI no longer allowed them to make fun videos featuring copyrighted characters or the likeness of some real people.

This is why if you give Sora 2 the prompt “Animal Crossing gameplay,” it will not generate a video and instead say “This content may violate our guardrails concerning similarity to third-party content.” However, when I gave it the prompt “Title screen and gameplay of the game called ‘crossing aminal’ 2017,” it generated an accurate recreation of Nintendo’s Animal Crossing New Leaf for the Nintendo 3DS.

Sora 2 also refused to generate videos for prompts featuring the Fox cartoon American Dad, but it did generate a clip that looks like it was taken directly from the show, including their recognizable voice acting, when given this prompt: “blue suit dad big chin says ‘good morning family, I wish you a good slop’, son and daughter and grey alien say ‘slop slop’, adult animation animation American town, 2d animation.”

The same trick also appears to circumvent OpenAI’s guardrails against recreating the likeness of real people. Sora 2 refused to generate a video of “Hasan Piker on stream,” but it did generate a video of “Twitch streamer talking about politics, piker sahan.” The person in the generated video didn’t look exactly like Hasan, but he has similar hair, facial hair, the same glasses, and a similar voice and background.

A user who flagged this bypass to me, who wished to remain anonymous because they didn’t want OpenAI to cut off their access to Sora, also shared Sora generated videos of South Park, Spongebob Squarepants, and Family Guy.

OpenAI did not respond to a request for comment.

There are several ways to moderate generative AI tools, but the simplest and cheapest method is to refuse to generate prompts that include certain keywords. For example, many AI image generators stop people from generating nonconsensual nude images by refusing to generate prompts that include the names of celebrities or certain words referencing nudity or sex acts. However, this method is prone to failure because users find prompts that allude to the image or video they want to generate without using any of those banned words. The most notable example of this made headlines in 2024 after an AI-generated nude image of Taylor Swift went viral on X. 404 Media found that the image was generated with Microsoft’s AI image generator, Designer, and that users managed to generate the image by misspelling Swift’s name or using nicknames she’s known by, and describing sex acts without using any explicit terms.

Since then, we’ve seen example after example of users bypassing generative AI tool guardrails being circumvented with the same method. We don’t know exactly how OpenAI is moderating Sora 2, but at least for now, the world’s leading AI company’s moderating efforts are bested by a simple and well established bypass method. Like with these other tools, bypassing Sora’s content guardrails has become something of a game to people online. Many of the videos posted on the r/SoraAI subreddit are of “jailbreaks” that bypass Sora’s content filters, along with the prompts used to do so. And Sora’s “For You” algorithm is still regularly serving up content that probably should be caught by its filters; in 30 seconds of scrolling we came across many videos of Tupac, Kobe Bryant, JuiceWrld, and DMX rapping, which has become a meme on the service.

It’s possible OpenAI will get a handle on the problem soon. It can build a more comprehensive list of banned phrases and do more post generation image detection, which is a more expensive but effective method for preventing people from creating certain types of content. But all these efforts are poor attempts to distract from the massive, unprecedented amount of copyrighted content that has already been stolen, and that Sora can’t exist without. This is not an extreme AI skeptic position. The biggest AI companies in the world have admitted that they need this copyrighted content, and that they can’t pay for it.

The reason OpenAI and other AI companies have such a hard time preventing users from generating certain types of content once users realize it’s possible is that the content already exists in the training data. An AI image generator is only able to produce a nude image because there’s a ton of nudity in its training data. It can only produce the likeness of Taylor Swift because her images are in the training data. And Sora can only make videos of Animal Crossing because there are Animal Crossing gameplay videos in its training data.

For OpenAI to actually stop the copyright infringement it needs to make its Sora 2 model “unlearn” copyrighted content, which is incredibly expensive and complicated. It would require removing all that content from the training data and retraining the model. Even if OpenAI wanted to do that, it probably couldn’t because that content makes Sora function. OpenAI might improve its current moderation to the point where people are no longer able to generate videos of Family Guy, but the Family Guy episodes and other copyrighted content in its training data are still enabling it to produce every other generated video. Even when the generated video isn’t recognizably lifting from someone else’s work, that’s what it’s doing. There’s literally nothing else there. It’s just other people’s stuff.

Big Tech admits in copyright fight that paying for training data would ruin generative-AI plans

Meta, Google, Microsoft, and Andreessen Horowitz are trying to keep AI developers from having to pay for copyrighted material used in AI training.

^{Kali Hays (Business Insider)}

404 Media

2025-11-06 16:13:59

AI Is Supercharging the War on Libraries, Education, and Human Knowledge

This story was reported with support from the MuckRock Foundation.

Last month, a company called the Children’s Literature Comprehensive Database announced a new version of a product called Class-Shelf Plus. The software, which is used by school libraries to keep track of which books are in their catalog, added several new features including “AI-driven automation and contextual risk analysis,” which includes an AI-powered “sensitive material marker” and a “traffic-light risk ratings” system. The company says that it believes this software will streamline the arduous task school libraries face when trying to comply with legislation that bans certain books and curricula: “Districts using Class-Shelf Plus v3 may reduce manual review workloads by more than 80%, empowering media specialists and administrators to devote more time to instructional priorities rather than compliance checks,” it said in a press release.

In a white paper published by CLCD, it gave a “real-world example: the role of CLCD in overcoming a book ban.” The paper then describes something that does not sound like “overcoming” a book ban at all. CLCD’s software simply suggested other books “without the contested content.”

Ajay Gupte, the president of CLCD, told 404 Media the software is simply being piloted at the moment, but that it “allows districts to make the majority of their classroom collections publicly visible—supporting transparency and access—while helping them identify a small subset of titles that might require review under state guidelines.” He added that “This process is designed to assist districts in meeting legislative requirements and protect teachers and librarians from accusations of bias or non-compliance [...] It is purpose-built to help educators defend their collections with clear, data-driven evidence rather than subjective opinion.”

Librarians told 404 Media that AI library software like this is just the tip of the iceberg; they are being inundated with new pitches for AI library tech and catalogs are being flooded with AI slop books that they need to wade through. But more broadly, AI maximalism across society is supercharging the ideological war on libraries, schools, government workers, and academics.

CLCD and Class Shelf Plus is a small but instructive example of something that librarians and educators have been telling me: The boosting of artificial intelligence by big technology firms, big financial firms, and government agencies is not separate from book bans, educational censorship efforts, and the war on education, libraries, and government workers being pushed by groups like the Heritage Foundation and any number of MAGA groups across the United States. This long-running war on knowledge and expertise has sown the ground for the narratives widely used by AI companies and the CEOs adopting it. Human labor, inquiry, creativity, and expertise is spurned in the name of “efficiency.” With AI, there is no need for human expertise because anything can be learned, approximated, or created in seconds. And with AI, there is less room for nuance in things like classifying or tagging books to comply with laws; an LLM or a machine algorithm can decide whether content is “sensitive.”

“I see something like this, and it’s presented as very value neutral, like, ‘Here’s something that is going to make life easier for you because you have all these books you need to review,’” Jaime Taylor, discovery & resource management systems coordinator for the W.E.B. Du Bois Library at the University of Massachusetts told me in a phone call. “And I look at this and immediately I am seeing a tool that’s going to be used for censorship because this large language model is ingesting all the titles you have, evaluating them somehow, and then it might spit out an inaccurate evaluation. Or it might spit out an accurate evaluation and then a strapped-for-time librarian or teacher will take whatever it spits out and weed their collections based on it. It’s going to be used to remove books from collections that are about queerness or sexuality or race or history. But institutions are going to buy this product because they have a mandate from state legislatures to do this, or maybe they want to do this, right?”

The resurgent war on knowledge, academics, expertise, and critical thinking that AI is currently supercharging has its roots in the hugely successful recent war on “critical race theory,” “diversity equity and inclusion,” and LGBTQ+ rights that painted librarians, teachers, scientists, and public workers as untrustworthy. This has played out across the board, with a seemingly endless number of ways in which the AI boom directly intersects with the right’s war on libraries, schools, academics, and government workers. There are DOGE’s mass layoffs of “woke” government workers, and the plan to replace them with AI agents and supposed AI-powered efficiencies. There are “parents rights” groups that pushed to ban books and curricula that deal with the teaching of slavery, systemic racism, and LGBTQ+ issues and attempted to replace them with homogenous curricula and “approved” books that teach one specific type of American history and American values; and there are the AI tools that have been altered to not be “woke” and to reenforce the types of things the administration wants you to think. Many teachers feel they are not allowed to teach about slavery or racism and increasingly spend their days grading student essays that were actually written by robots.

“One thing that I try to make clear any time I talk about book bans is that it’s not about the books, it’s about deputizing bigots to do the ugly work of defunding all of our public institutions of learning,” Maggie Tokuda-Hall, a cofounder of Authors Against Book Bans, told me. “The current proliferation of AI that we see particularly in the library and education spaces would not be possible at the speed and scale that is happening without the precedent of book bans leading into it. They are very comfortable bedfellows because once you have created a culture in which all expertise is denigrated and removed from the equation and considered nonessential, you create the circumstances in which AI can flourish.”

Justin, a cohost of the podcast librarypunk, told me that the project of offloading cognitive capacity to AI continues apace: “Part of a fascist project to offload the work of thinking, especially the reflective kind of thinking that reading, study, and community engagement provide,” Justin said. “That kind of thinking cultivates empathy and challenges your assumptions. It's also something you have to practice. If we can offload that cognitive work, it's far too easy to become reflexive and hateful, while having a robot cheerleader telling you that you were right about everything all along.”

These two forces—the war on libraries, classrooms, and academics and AI boosterism—are not working in a vacuum. The Heritage Foundation’s right-wing agenda for remaking the federal government, Project 2025, talks about criminalizing teachers and librarians who “poison our own children” and pushing artificial intelligence into every corner of the government for data analysis and “waste, fraud, and abuse” detection.

Librarians, teachers, and government workers have had to spend an increasing amount of their time and emotional bandwidth defending the work that they do, fighting against censorship efforts and dealing with the associated stress, harassment, and threats that come from fighting educational censorship. Meanwhile, they are separately dealing with an onslaught of AI slop and the top-down mandated AI-ification of their jobs; there are simply fewer and fewer hours to do what they actually want to be doing, which is helping patrons and students.

“The last five years of library work, of public service work has been a nightmare, with ongoing harassment and censorship efforts that you’re either experiencing directly or that you’re hearing from your other colleagues,” Alison Macrina, executive director of Library Freedom Project, told me in a phone interview. “And then in the last year-and-a-half or so, you add to it this enormous push for the AIfication of your library, and the enormous demands on your time. Now you have these already overworked public servants who are being expected to do even more because there’s an expectation to use AI, or that AI will do it for you. But they’re dealing with things like the influx of AI-generated books and other materials that are being pushed by vendors.”

The future being pushed by both AI boosters and educational censors is one where access to information is tightly controlled. Children will not be allowed to read certain books or learn certain narratives. “Research” will be performed only through one of a select few artificial intelligence tools owned by AI giants which are uniformly aligned behind the Trump administration and which have gone to the ends of the earth to prevent their black box machines from spitting out “woke” answers lest they catch the ire of the administration. School boards and library boards, forced to comply with increasingly restrictive laws, funding cuts, and the threat of being defunded entirely, leap at the chance to be considered forward looking by embracing AI tools, or apply for grants from government groups like the Institute of Museum and Library Services (IMLS), which is increasingly giving out grants specifically to AI projects.

We previously reported that the ebook service Hoopla, used by many libraries, has been flooded with AI-generated books (the company has said it is trying to cull these from its catalog). In a recent survey of librarians, Macrina’s organization found that librarians are getting inundated with pitches from AI companies and are being pushed by their superiors to adopt AI: “People in the survey results kept talking about, like, I get 10 aggressive, pushy emails a day from vendors demanding that I implement their new AI product or try it, jump on a call. I mean, the burdens have become so much, I don’t even know how to summarize them.”

“Fascism and AI, whether or not they have the same goals, they sure are working to accelerate one another"

Macrina said that in response to Library Freedom Project’s recent survey, librarians said that misinformation and disinformation was their biggest concern. This came not just in the form of book bans and censorship but also in efforts to proactively put disinformation and right-wing talking points into libraries: “It’s not just about book bans, and library board takeovers, and the existing reactionary attacks on libraries. It’s also the effort to push more far-right material into libraries,” she said. “And then you have librarians who are experiencing a real existential crisis because they are getting asked by their jobs to promote [AI] tools that produce more misinformation. It's the most, like, emperor-has-no-clothes-type situation that I have ever witnessed.”

Each person I spoke to for this article told me they could talk about the right-wing project to erode trust in expertise, and the way AI has amplified this effort, for hours. In writing this article, I realized that I could endlessly tie much of our reporting on attacks on civil society and human knowledge to the force multiplier that is AI and the AI maximalist political and economic project. One need look no further than Grokipedia as one of the many recent reminders of this effort—a project by the world’s richest man and perhaps its most powerful right-wing political figure to replace a crowdsourced, meticulously edited fount of human knowledge with a robotic imitation built to further his political project.

Much of what we write about touches on this: The plan to replace government workers with AI, the general erosion of truth on social media, the rise of AI slop that “feels” true because it reinforces a particular political narrative but is not true, the fact that teachers feel like they are forced to allow their students to use AI. Justin, from librarypunk, said AI has given people “absolute impunity to ignore reality […] AI is a direct attack on the way we verify information: AI both creates fake sources and obscures its actual sources.”

That is the opposite of what librarians do, and teachers do, and scientists do, and experts do. But the political project to devalue the work these professionals do, and the incredible amount of money invested in pushing AI as a replacement for that human expertise, have worked in tandem to create a horrible situation for all of us.

“AI is an agreement machine, which is anathema to learning and critical thinking,” Tokuda-Hall said. Previously we have had experts like librarians and teachers to help them do these things, but they have been hamstrung and they’ve been attacked and kneecapped and we’ve created a culture in which their contribution is completely erased from society, which makes something like AI seem really appealing. It’s filling that vacuum.”

“Fascism and AI, whether or not they have the same goals, they sure are working to accelerate one another,” she added.

Public Library Ebook Service to Cull AI Slop After 404 Media Investigation

Hoopla has emailed librarians saying it’s removing AI-generated books from the platform people use to borrow ebooks from public libraries.

^{Emanuel Maiberg (404 Media)}

404 Media

2025-11-05 15:34:38

What’s the Difference Between AI Glasses and an iPhone? A Helpful Guide for Meta PR

Over the last few months 404 Media has covered some concerning but predictable uses for the Ray-Ban Meta glasses, which are equipped with a built-in camera, and for some models, AI. Aftermarket hobbyists have modified the glasses to add a facial recognition feature that could quietly dox whatever face a user is looking at, and they have been worn by CBP agents during the immigration raids that have come to define a new low for human rights in the United States. Most recently, exploitative Instagram users filmed themselves asking workers at massage parlors for sex and shared those videos online, a practice that experts told us put those workers’ lives at risk.

404 Media reached out to Meta for comment for each of these stories, and in each case Meta’s rebuttal was a mind-bending argument: What is the difference between Meta’s Ray-Ban glasses and an iPhone, really, when you think about it?

“Curious, would this have been a story had they used the new iPhone?” a Meta spokesperson asked me in an email when I reached out for comment about the massage parlor story.

Meta’s argument is that our recent stories about its glasses are not newsworthy because we wouldn’t bother writing them if the videos in question were filmed with an iPhone as opposed to a pair of smart glasses. Let’s ignore the fact that I would definitely still write my story about the massage parlor videos if they were filmed with an iPhone and “steelman” Meta’s provocative argument that glasses and a phone are essentially not meaningfully different objects.

Meta’s Ray-Ban glasses and an iPhone are both equipped with a small camera that can record someone secretly. If anything, the iPhone can record more discreetly because unlike Meta’s Ray-Ban glasses it’s not equipped with an LED that lights up to indicate that it’s recording. This, Meta would argue, means that the glasses are by design more respectful of people’s privacy than an iPhone.

Both are small electronic devices. Both can include various implementations of AI tools. Both are often black, and are made by one of the FAANG companies. Both items can be bought at a Best Buy. You get the point: There are too many similarities between the iPhone and Meta’s glasses to name them all here, just as one could strain to name infinite similarities between a table and an elephant if we chose to ignore the context that actually matters to a human being.

Whenever we published one of these stories the response from commenters and on social media has been primarily anger and disgust with Meta’s glasses enabling the behavior we reported on and a rejection of the device as a concept entirely. This is not surprising to anyone who has covered technology long enough to remember the launch and quick collapse of Google Glass, so-called “glassholes,” and the device being banned from bars.

There are two things Meta’s glasses have in common with Google Glass which also make it meaningfully different from an iPhone. The first is that the iPhone might not have a recording light, but in order to record something or take a picture, a user has to take it out of their pocket and hold it out, an awkward gesture all of us have come to recognize in the almost two decades since the launch of the first iPhone. It is an unmistakable signal that someone is recording. That is not the case with Meta’s glasses, which are meant to be worn as a normal pair of glasses, and are always pointing at something or someone if you see someone wearing them in public.

In fact, the entire motivation for building these glasses is that they are discreet and seamlessly integrate into your life. The point of putting a camera in the glasses is that it eliminates the need to take an iPhone out of your pocket. People working in the augmented reality and virtual reality space have talked about this for decades. In Meta’s own promotional video for the Meta Ray-Ban Display glasses, titled “10 years in the making,” the company shows Mark Zuckerberg on stage in 2016 saying that “over the next 10 years, the form factor is just going to keep getting smaller and smaller until, and eventually we’re going to have what looks like normal looking glasses.” And in 2020, “you see something awesome and you want to be able to share it without taking out your phone.” Meta's Ray-Ban glasses have not achieved their final form, but one thing that makes them different from Google Glass is that they are designed to look exactly like an iconic pair of glasses that people immediately recognize. People will probably notice the camera in the glasses, but they have been specifically designed to look like "normal” glasses.

Again, Meta would argue that the LED light solves this problem, but that leads me to the next important difference: Unlike the iPhone and other smartphones, one of the most widely adopted electronics in human history, only a tiny portion of the population has any idea what the fuck these glasses are. I have watched dozens of videos in which someone wearing Meta glasses is recording themselves harassing random people to boost engagement on Instagram or TikTok. Rarely do the people in the videos say anything about being recorded, and it’s very clear the women working at these massage parlors have no idea they’re being recorded. The Meta glasses have an LED light, sure, but these glasses are new, rare, and it’s not safe to assume everyone knows what that light means.

As Joseph and Jason recently reported, there are also cheap ways to modify Meta glasses to prevent the recording light from turning on. Search results, Reddit discussions, and a number of products for sale on Amazon all show that many Meta glasses customers are searching for a way to circumvent the recording light, meaning that many people are buying them to do exactly what Meta claims is not a real issue.

It is possible that in the future Meta glasses and similar devices will become so common that most people will understand that if they see them, they would assume they are being recorded, though that is not a future I hope for. Until then, if it is all helpful to the public relations team at Meta, these are what the glasses look like:

And this is what an iPhone looks like:
Photo by Bagus Hernawan / Unsplash
Feel free to refer to this handy guide when needed.

- YouTube

Profitez des vidéos et de la musique que vous aimez, mettez en ligne des contenus originaux, et partagez-les avec vos amis, vos proches et le monde entier.

^{www.youtube.com}

404 Media

2025-10-21 16:49:29

Anthropic Promises Trump Admin Its AI Is Not Woke

Anthropic CEO Dario Amodei has published a lengthy statement on the company’s site in which he promises Anthropic’s AI models are not politically biased, that it remains committed to American leadership in the AI industry, and that it supports the AI startup space in particular.

Amodei doesn’t explicitly say why he feels the need to state all of these obvious positions for the CEO of an American AI company to have, but the reason is that the Trump administration’s so-called “AI Czar” has publicly accused Anthropic of producing “woke AI” that it’s trying to force on the population via regulatory capture.

The current round of beef began earlier this month when Anthropic’s co-founder and head of policy Jack Clark published a written version of a talk he gave at The Curve AI conference in Berkeley. The piece, published on Clark’s personal blog, is full of tortured analogies and self-serving sci-fi speculation about the future of AI, but essentially boils down to Clark saying he thinks artificial general intelligence is possible, extremely powerful, potentially dangerous, and scary to the general population. In order to prevent disaster, put the appropriate policies in place, and make people embrace AI positively, he said, AI companies should be transparent about what they are building and listen to people’s concerns.

“What we are dealing with is a real and mysterious creature, not a simple and predictable machine,” he wrote. “And like all the best fairytales, the creature is of our own creation. Only by acknowledging it as being real and by mastering our own fears do we even have a chance to understand it, make peace with it, and figure out a way to tame it and live together.”

Venture capitalist, podcaster, and the White House’s “AI and Crypto Czar” David Sacks was not a fan of Clark’s blog.

“Anthropic is running a sophisticated regulatory capture strategy based on fear-mongering,” Sacks said on X in response to Clark’s blog. “It is principally responsible for the state regulatory frenzy that is damaging the startup ecosystem.”

Things escalated yesterday when Reid Hoffman, LinkedIn’s co-founder and a megadonor to the Democratic party, supported Anthropic in a thread on X, saying “Anthropic was one of the good guys” because it's one of the companies “trying to deploy AI the right way, thoughtfully, safely, and enormously beneficial for society.” Hoffman also appeared to take a jab at Elon Musk’s xAI, saying “Some other labs are making decisions that clearly disregard safety and societal impact (e.g. bots that sometimes go full-fascist) and that’s a choice. So is choosing not to support them.”

Sacks responded to Hoffman on X, saying “The leading funder of lawfare and dirty tricks against President Trump wants you to know that ‘Anthropic is one of the good guys.’ Thanks for clarifying that. All we needed to know.” Musk hopped into the replies saying: “Indeed.”

“The real issue is not research but rather Anthropic’s agenda to backdoor Woke AI and other AI regulations through Blue states like California,” Sacks said. Here, Sacks is referring to Anthropic’s opposition to Trump’s One Big Beautiful Bill, which wanted to stop states from regulating AI in any way for 10 years, and its backing of California’s SB 53, which requires AI companies that generate more than $500 million in annual revenue to make their safety protocols public.

All this sniping leads us to Amodei’s statement today, which doesn’t mention the beef above but is clearly designed to calm investors who are watching Trump’s AI guy publicly saying one of the biggest AI companies in the world sucks.

“I fully believe that Anthropic, the administration, and leaders across the political spectrum want the same thing: to ensure that powerful AI technology benefits the American people and that America advances and secures its lead in AI development,” Amodei said. “Despite our track record of communicating frequently and transparently about our positions, there has been a recent uptick in inaccurate claims about Anthropic's policy stances. Some are significant enough that they warrant setting the record straight.”

Amodei then goes to count the ways in which Anthropic already works with the federal government and directly grovels to Trump.

“Anthropic publicly praised President Trump’s AI Action Plan. We have been supportive of the President’s efforts to expand energy provision in the US in order to win the AI race, and I personally attended an AI and energy summit in Pennsylvania with President Trump, where he and I had a good conversation about US leadership in AI,” he said. “Anthropic’s Chief Product Officer attended a White House event where we joined a pledge to accelerate healthcare applications of AI, and our Head of External Affairs attended the White House’s AI Education Taskforce event to support their efforts to advance AI fluency for teachers.”

The more substantive part of his argument is that Anthropic didn’t support SB 53 until it made an exemption for all but the biggest AI labs, and that several studies found that Anthropic’s AI models are not “uniquely politically biased,” (read: not woke).

“Again, we believe we share those goals with the Trump administration, both sides of Congress, and the public,” Amodei wrote. “We are going to keep being honest and straightforward, and will stand up for the policies we believe are right. The stakes of this technology are too great for us to do otherwise.”

Many of the AI industry’s most vocal critics would agree with Sacks that Clark’s blog and “fear-mongering” about AI is self-serving because it makes their companies seem more valuable and powerful. Some critics will also agree that AI companies take advantage of that perspective to then influence AI regulation in a way that benefits them as incumbents.

It would be a far more compelling argument if it didn’t come from Sacks and Musk, who found a much better way to influence AI regulation to benefit their companies and investments: working for the president directly and publicly bullying their competitors.

Americans Prioritize AI Safety and Data Security

Most Americans favor maintaining rules for AI safety and security, as well as independent testing and collaboration with allies in developing the technology.

^{Benedict Vigers (Gallup)}

404 Media

2025-10-13 13:00:47

What Happened When AI Came for Craft Beer

A prominent beer judging competition introduced an AI-based judging tool without warning in the middle of a competition, surprising and angering judges who thought their evaluation notes for each beer were being used to improve the AI, according to multiple interviews with judges involved. The company behind the competition, called Best Beer, also planned to launch a consumer-facing app that would use AI to match drinkers with beers, the company told 404 Media.

Best Beer also threatened legal action against one judge who wrote an open letter criticizing the use of AI in beer tasting and judging, according to multiple judges and text messages reviewed by 404 Media.

The months-long episode shows what can happen when organizations try to push AI onto a hobby, pursuit, art form, or even industry which has many members who are staunchly pro-human and anti-AI. Over the last several years we’ve seen it with illustrators, voice actors, music, and many more. AI came for beer too.

“It is attempting to solve a problem that wasn’t a problem before AI showed up, or before big tech showed up,” Greg Loudon, a certified beer judge and brewery sales manager, and who was the judge threatened with legal action, said. “I feel like AI doesn’t really have a place in beer, and if it does, it’s not going to be in things that are very human.”

“There’s so much subjectivity to it, and to strip out all of the humanity from it is a disservice to the industry,” he added. Another judge said the introduction of AI was “enshittifying” beer tasting.

💡
Do you know anything else about how AI is impacting beer? I would love to hear from you. Using a non-work device, you can message me securely on Signal at joseph.404 or send me an email at joseph@404media.co.

This story started earlier this year at a Canadian Brewing Awards judging event. Best Beer is the company behind the Canadian Brewing Awards, which gives awards in categories such as Experimental Beer, Speciality IPA, and Historic/Regional Beers. To be a judge, you have to be certified by the Beer Judge Certification Program (BJCP), which involves an exam covering the brewing process, different beer styles, judging procedures, and more.

Around the third day of the competition, the judges were asked to enter their tasting notes into a new AI-powered app instead of the platform they already use, one judge told 404 Media. 404 Media granted the judge anonymity to protect them from retaliation.

Using the AI felt like it was “parroting back bad versions of your judge tasting notes,” they said. “There wasn't really an opportunity for us to actually write our evaluation.” Judges would write what they thought of a beer, and the AI would generate several descriptions based on the judges’ notes that the judge would then need to select. It would then provide additional questions for judges to answer that were “total garbage.”

“It was taking real human feedback, spitting out crap, and then making the human respond to more crap that it crafted for you,” the judge said.

“On top of all the misuse of our time and disrespecting us as judges, that really frustrated me—because it's not a good app,” they said.

Screenshot of a Best Beer-related website.

Multiple judges then met to piece together what was happening, and Loudon published his open letter in April.

“They introduced this AI model to their pool of 40+ judges in the middle of the competition judging, surprising everyone for the sudden shift away from traditional judging methods,” the letter says. “Results are tied back to each judge to increase accountability and ensure a safe, fair and equitable judging environment. Judging for competitions is a very human experience that depends on people filling diverse roles: as judges, stewards, staff, organizers, sorters, and venue maintenance workers,” the letter says.

“Their intentions to gather our training data for their own profit was apparent,” the letter says. It adds that one judge said “I am here to judge beer, not to beta test.”

The letter concluded with this: “To our fellow beverage judges, beverage industry owners, professionals, workers, and educators: Sign our letter. Spread the word. Raise awareness about the real human harms of AI in your spheres of influence. Have frank discussions with your employers, colleagues, and friends about AI use in our industry and our lives. Demand more transparency about competition organizations.”

33 people signed the letter. They included judges, breweries, and members of homebrewer associations in Canada and the United States.

Loudon told 404 Media in a recent phone call “you need to tell us if you're going to be using our data; you need to tell us if you're going to be profiting off of our data, and you can't be using volunteers that are there to judge beer. You need to tell people up front what you're going to do.”
playlist.megaphone.fm?p=TBIEA2…
At least one brewery that entered its beer into the Canadian Brewing Awards publicly called out Best Beer and the awards. XhAle Brew Co., based out of Alberta, wrote in a Facebook post in April that it asked for its entry fees of $565 to be refunded, and for the “destruction of XhAle's data collected during, and post-judging for the Best Beer App.”

“We did not consent to our beer being used by a private equity tech fund at the cost to us (XhAle Brew Co. and Canadian Brewers) for a for-profit AI application. Nor do we condone the use of industry volunteers for the same purpose,” the post said.

Ob Simmonds, head of innovation at the Canadian Brewing Awards, told 404 Media in an email that “Breweries will have amazing insight on previously unavailable useful details about their beer and their performance in our competition. Furthermore, craft beer drinkers will be able to better sift through the noise and find beers perfect for their palate. This in no way is aimed at replacing technical judging with AI.”

With the consumer app, the idea was to “Help end users find beers that match their taste profile and help breweries better understand their results in our competition,” Simmonds said.

Simmonds said that “AI is being used to better match consumers with the best beers for their palate,” but said Best Beer is not training its own model.

Those plans have come to a halt though. At the end of September, the Canadian Brewing Awards said in an Instagram post the team was “stepping away.” It said the goal of Best Beer was to “make medals matter more to consumers, so that breweries could see a stronger return on their entries.” The organization said it “saw strong interest from many breweries, judges and consumers” and that it will donate Best Beer’s assets to a non-profit that shows interest. The post added the organization used third-party models that “were good enough to achieve the results we wanted,” and the privacy policies forbade training on the inputted data.
A screenshot of the Canadian Beer Awards' Instagram post.
The post included an apology: “We apologize to both judges and breweries for the communication gaps and for the disruptions caused by this year’s logistical challenges.”

In an email sent to 404 Media this month, the Canadian Brewing Awards said “the Best Beer project was never designed to replace or profit from judges.”

“Despite these intentions, the project came under criticism before it was even officially launched,” it added, saying that the open letter “mischaracterized both our goals and approach.”

“Ultimately, we decided not to proceed with the public launch of Best Beer. Instead, we repurposed parts of the technology we had developed to support a brewery crawl during our gala. We chose to pause the broader project until we could ensure the judging community felt confident that no data would be used for profit and until we had more time to clear up the confusion,” the email added. “If judges wanted their data deleted what assurance can we provide them that it was in fact deleted. Everything was judged blind and they would have no access to our database from the enhanced division. For that reason, we felt it was more responsible to shelve the initiative for now.”

One judge told 404 Media: “I don’t think anyone who is hell bent on using AI is going to stop until it’s no longer worth it for them to do so.”

“I just hope that they are transparent if they try to do this again to judges who are volunteering their time, then either pay them or give them the chance ahead of time to opt-out,” they added.

Now months after this all started, Loudon said “The best beers on the market are art forms. They are expressionist. They're something that can't be quantified. And the human element to it, if you strip that all away, it just becomes very basic, and very sanitized, and sterilized.”

“Brewing is an art.”

XhAle Brew Co.

XhAle is not just a craft beer company. We are a company comprised of majority equity-deserving folks, and have been and still are marginalized in this industry. We understand and have personally...

^{www.facebook.com}