404 Media

2026-03-26 16:13:09

Wikipedia Bans AI-Generated Content

After months of heated debate and previous attempts to restrict the use of large language models on Wikipedia, on March 20 volunteer editors accepted a new policy that prohibits using them to create articles for the online encyclopedia.

“Text generated by large language models (LLMs) often violates several of Wikipedia's core content policies,” Wikipedia’s new policy states. “For this reason, the use of LLMs to generate or rewrite article content is prohibited, save for the exceptions given below.”

The new policy, which was accepted in an overwhelming 40 to 2 vote among editors, allows editors to use LLMs to suggest basic copyedits to their own writing, which can be incorporated into the article or rewritten after human review if the LLM doesn’t generate entirely new content on its own.

“Caution is required, because LLMs can go beyond what you ask of them and change the meaning of the text such that it is not supported by the sources cited,” the policy states. “The use of LLMs to translate articles from another language's Wikipedia into the English Wikipedia must follow the guidance laid out at Wikipedia:LLM-assisted translation.”

I previously reported about editors using LLMs to translate Wikipedia articles and introducing errors to those articles in the process.

Wikipedia editor, Ilyas Lebleu, who goes by Chaotic Enby on Wikipedia and who proposed the guideline said that it seemed unlikely the policy will last because previously the editor community has been divided on the issue. However, Lebleu said “The mood was shifting, with holdouts of cautious optimism turning to genuine worry.”

“A few months ago, a much more bare-bones guideline had passed, only banning the creation of brand new articles with LLMs,” Lebleu told me in an email. “A follow-up proposal to reword it into something more substantial failed to pass, but was noted to have ‘consensus for better guidelines along the lines of and/or in the spirit of this draft.’ In recent months, more and more administrative reports centered on LLM-related issues, and editors were being overwhelmed.”

The policy was written with the help of WikiProject AI Cleanup, a group of Wikipedia editors dedicated to finding and removing AI-generated errors on the site. Editors have been dealing with an increasing number of AI-generated articles or edits lately, and have made some minor adjustments to its guidelines as a result, like streamlining the process for removing AI-generated articles. Editors’ position, as well as the position of the Wikimedia Foundation, has been to not make blanket rules against AI because Wikipedia already uses some forms of automation, and because AI tools could assist editors in the future.

The new policy doesn’t ban the use of other automated tools that are already in use or future implementations, but it does show the Wikipedia community is less optimistic about the benefit of AI-generated content, and taking a stand against it.

“In context, this has implications far beyond Wikipedia,” Lebleu said. “The same flood of AI-generated content has been seen from social media to open-source projects, where agents submit pull requests much faster than human reviewers can keep up with. StackOverflow and the German Wikipedia paved the way in recent months with similar policies, and, as anxiety over the AI bubble grows, I foresee a domino effect, empowering communities on other platforms to decide whether AI should be welcome. On their own terms.”

Wikipedia Editors Adopt ‘Speedy Deletion’ Policy for AI Slop Articles

“The ability to quickly generate a lot of bogus content is problematic if we don't have a way to delete it just as quickly.”

^{Emanuel Maiberg (404 Media)}

404 Media

2026-03-24 13:35:37

A Top Google Search Result for Claude Plugins Was Planted by Hackers

A top result on Google for people searching for Claude plugins sent users to a site that recently contained malicious code in an apparent attempt to steal their credentials.

The news shows how the explosion of interest in generative AI tools is giving hackers new ways to attack users.

The malicious site was flagged to us by a 404 Media reader who was using Claude.

“I was googling to troubleshoot how to get my Claude Code CLI to authenticate its github plugin to my Github account and may have stumbled upon a malicious site hosted on Squarespace of all places,” the reader, Dan Foley, told me in an email.

Foley searched for “github plugin claude code” and the top result was a sponsored ad for a Squarespace site with the title “Install Claude Code - Claude Code Docs.”

When he clicked through, he saw a site that was pretending to be the official site for Anthropic’s Claude with identical design and branding.

The phony Anthropic help site had swapped some of the Claude Code installation instructions for others, Foley pointed out. That included a line users could paste into their terminal to allegedly install the software on a Mac. The command included an obfuscated URL, hiding what its real destination was. When Foley decoded it, he found it downloaded software from another site entirely.

ThreatFox, a platform for sharing known instances of malware, recently flagged that domain as sharing a “stealer”, a type of malware that steals users credentials. ThreatFox linked that domain to the stealer as recently as a few days ago.

Google’s ad center listed the advertiser behind the malicious sponsored search result as “Enhancv R&D,” which is based in Bulgaria, according to a screenshot of the advertiser profile Foley shared with 404 Media. The advertiser was also listed as being verified by Google, meaning they had to complete an identity verification process which requires legal documentation of their name and location.

Foley said he flagged the ad to Google, which removed the site from search results. The URL which pointed to the potential stealer is no longer online.

“We removed this ad and suspended the account for violating our policies,” a Google spokesperson told me in an email. Google said it has strict policies against ads that aim to phish information or distribute malware, and that it uses a combination of Gemini-powered tools and human review to enforce these policies at scale. Google claims the vast majority of these ads are caught before the ads ever run.

Malicious links included in paid Google ads that are pretending to be legitimate websites is not a problem that’s unique to AI. Hackers often try to get users to click malicious links by pretending to be whatever is popular on the internet at any given moment, be it a pirated movie or video game just before release or celebrity sex tapes. The fact that hackers are targeting Claude users reflects the growing popularity of AI tools and the hackers’ hope that users are not careful enough to check what they’re clicking when using them.

In January, we wrote about how hackers could similarly target users of the AI agent tool OpenClaw by boosting instructions for AI agents that contained a backdoor for hackers.

"GoogleFix" - Hijacking Google's Sponsored Results to Deliver Infostealers

ClickFix's Latest Evolution Now Hijacks Google's Sponsored Results Infected with AMOS Stealer Lures

^{Guardio Research Team (Guardio)}

404 Media

2026-03-23 15:20:26

This Web Tool Sabotages AI Chatbots By Making Them Really, Really Slow

Watching people outsource their critical thinking, emotions, and sanity to glitchy “AI” chatbots has been one of the most uniquely terrifying aspects of being a human being in recent years.

While wealthy tech evangelists like Sam Altman continue to make wild proclamations about how large language models (LLMs) are destined to do our jobs and raise our children, critics have compared Silicon Valley’s attempts to force dependence on chatbots to a mass-enfeebling event—an attempt to convince people that they are actually better off having machines think, act, and create for them.

Now, there’s a new way to discourage friends, family, and even complete strangers from turning to chatbots like Claude and ChatGPT: by using a tool called “Slow LLM” to make them really, reaaaaalllyyy slowwwww. Or at least, making them look that way.

“Are you concerned that you or your loved ones might be participating in a massive de-skilling event? Experiencing LLM-induced psychosis? Outsourcing cognitive and emotional functions to autocomplete? Install SLOW LLM on your computer, or the computer of a loved one, today!” reads a description onthe tool’s website.

Created by artist Sam Lavigne, Slow LLM causes anyone accessing AI chatbots on a computer or network to encounter mysterious, painfully slow response times. It works by manipulating a quirk in the Javascript language to rewrite the “Fetch” function that returns data to the browser. When a user visits a chatbot domain and enters a query, the modified Fetch function stretches the response over an excruciatingly long period of time. This results in the user perceiving the LLM to be running slowly, when in reality it’s simply being arbitrarily metered by Lavigne’s code.

Lavigne says that the idea for the project came after seeing how deeply some of his students and acquaintances had come to rely on generative tools to do basic tasks.

“So many people are starting to use these tools to outsource their cognitive and emotional functions, and in the process of doing this they’re forgetting all these basic things that they’ve learned how to do,” Lavigne told 404 Media. “I think that the more people rely on LLMs, the more extreme this de-skilling event will become.”

Slow LLM can be installed as aChrome browser extension, but it can also be deployed network-wide via an “Enterprise Edition,” aDNS service which causes everyone on a home, school, or corporate network to experience slow chatbot responses. This is done by simplychanging the DNS server on your router to Lavigne’s custom domain—though he warns that using a random person’s DNS is generally not a great idea cybersecurity-wise, and recommends the safer option ofhosting your own DNS server to deploy theSlow LLM code, which he has released for free on Github. The browser extension currently only affects Claude and ChatGPT, while the DNS version also slows down Grok and Google Gemini.

“The idea was that these things are removing friction, so let’s add some friction back in,” said Lavigne, using the engineering term frequently used by tech bros to describe inefficiencies in a system. He argues that LLM chatbots have taken this idea of “friction” to an extreme, presenting any unpleasantness or difficulty we encounter as something that should be outsourced to Silicon Valley’s thinking machines—even if overcoming that difficulty is part of what makes human creativity meaningful and worthwhile. “Anything that removes the friction of something that’s difficult, it makes you not learn, and it removes the learning you’ve already achieved.”

In theory, one could activate Slow LLM without anyone noticing; most people would likely assume that chatbot providers like Google and OpenAI are having technical issues, which does happen without outside interference from time to time. Lavigne says that so far, he hasn’t heard from anyone that has successfully deployed Slow LLM on a work or school network. But he certainly isn’t discouraging people from trying.

“I have not yet tested it on any unwitting subjects, but I’m thinking about it,” Lavigne said in a mischievous tone, adding that it would be an interesting experiment to see how people react when presented with artificially-slow chatbots. “Maybe they’ll just rage-quit LLMs.”

Slow LLM is the latest addition to a series of impish tech provocations that Lavigne has become known for. During the height of the pandemic Zoompocalypse in 2021, he released “Zoom Escaper,” a tool that floods your Zoom audio stream with annoying echoes, distortions, and interruptions until your presence becomes unbearable to others. In 2018, he infamously scraped public LinkedIn profiles to build a massive database of ICE agents, which was subsequentlyremoved from platforms like Github and Medium. Lavigne’s frequent collaborator Tega Brain has also released browser tools like “Slop Evader,” whichfilters out generative AI slop by removing all search results from after November 2022, when ChatGPT was first released to the public.

“I’ve been doing these little experiments in digital sabotage where I’m trying to make these tools that mildly interrupt computational systems,” said Lavigne. “One of the things I’ve been thinking about is how if the means of production is truly in our hands, and it’s also the way we’re communicating with other people and managing our social life, then what does it mean to interrupt productivity?”

Lavigne is not an absolutist, however. Without prompting, he admitted that he used Claude to help write some of the code for Slow LLM—until, of course, Slow LLM started working and forced him to complete the project on his own. Instead, Lavigne says he’s trying to make people question the habits they are forming by regularly using chatbots, tools which tempt us to essentially entrust all our knowledge, decision-making, and emotional well-being to massive companies run by tech billionaires like Altman and Elon Musk.

“My hope is to get people to think a little bit more about their usage of these tools,” said Lavigne. “But the broader thing I want people to think about […] is ways of interrupting these flows of data, these flows of power, and putting friction into these computational systems that are mediating so many parts of our lives.”

'Slop Evader' Lets You Surf the Web Like It’s 2022

Artist Tega Brain is fighting the internet’s enshittification by turning back the clock to before ChatGPT existed.

^{Janus Rose (404 Media)}

404 Media

2026-03-20 13:50:54

Tiny Township Fears Iran Drone Strikes Because of New Nuclear Weapons Datacenter

The tiny township of Ypsilanti, Michigan, is worried about being a target for drone strikes thanks to a planned datacenter that the University of Michigan is building to support nuclear weapons research According to Douglas Winters, the township’s attorney, the University and Los Alamos National Laboratories (LANL) “have put a big bulls eye target on this entire township […] I believe it’s the truth.”

Winters delivered a report to the town’s Board of Trustees about the proposed datacenter during a public meeting on Tuesday. “Los Alamos, which produces the nuclear weapons, is a high value target,” he said. He pointed to America’s war in Iran as proof that the datacenter would be a target, noting that Iran’s drones had disabled AWS servers in the Middle East. “This is not a commercial datacenter. A Los Alamos datacenter is going to be the brains of the operation for nuclear modeling, nuclear weaponry.”
playlist.megaphone.fm?p=TBIEA2…
The university and LANL first announced their plan to build a $1.25 billion datacenter in 2024. The university picked nearby Ypsilanti Township—population of about 20,000—as the location for the datacenter and residents have been fighting it ever since. Concerns from the community are typical for people fighting against a datacenter: water, rising electricity bills, pollution, and noise.

Unique to the Ypsilanti datacenter fight, however, is its role in the production of nuclear weapons. The datacenter would service LANL, the birthplace of the atomic bomb and home to America’s nuclear weapons scientists. In January, LANL confirmed that the datacenter would, indeed, be used in nuclear weapons research.

To hear the university tell it, the datacenter will be one of the most advanced computing systems in the world. “We were told at the very beginning by U of M’s Vice President of public relations […] that they were going to build, in his words, the biggest, baddest, fastest computers in the world,” Winters said at the public meeting. “That, in of itself, is what makes these datacenters high value targets […] these data centers constitute power. Artificial intelligence is power. Supercomputers are power. And when something becomes that important, it becomes a target.”

View this post on Instagram

A post shared by 404 Media (@404mediaco)

Winters questioned the American military’s ability to protect targets from the threat of drone attacks on its own soil. “The drone capability is not a joke, folks,” he said. “The United States and Israel, in spite of all their high technology they’re bringing to bear in their war on Iran, they’ve actually had to request that Ukraine send their top advisors to help them understand how to best detect and destroy these drone attacks.”

He also questioned U of M’s values. Following a demand from the White House, the university eliminated its DEI programs in 2025. In February, again at the behest of the federal government, it announced the end of the PhD Project which helped people from underrepresented backgrounds get PhDs. “You have a situation now where the University of Michigan […] has cut a deal with the Department of War under Trump,” Winters said. “That’s what the University of Michigan has turned into by basically selling their soul to the Department of War.”

Jay Coghlan, the executive director of Nuclear Watch New Mexico, told 404 Media, “That LANL datacenter is going to be the brains for nuclear modeling and nuclear weaponry. Ultimately that's what it’s all about. Beware, a recent study found that in war games artificial intelligence went to escalation and nuclear war 95 percent of the time.”

According to Coghlan, the construction of the datacenter followed a familiar pattern. “The Lab has colonized brown people for eight decades here just like it’s now trying to do in Ypsilanti (New Mexico is 50 percent Hispanic and 12 percent Native American). But what the brown people in Ypsilanti have that they don’t have here is lots of water,” he told 404 Media.

Another topic of discussion at the Tuesday meeting was how to stop the construction of the datacenter. Winters and others explained that it’s been difficult to get the university, county, and other government powers to engage with them. Interested parties plead ignorance or recuse themselves because of financial involvement with U of M. “They’ve acted like The Godfather, making you an offer that you can’t refuse,” Winters said.

View this post on Instagram

A post shared by 404 Media (@404mediaco)

Trustee Karen Lovejoy Roe questioned why LANL wanted to build a datacenter 1,500 miles away from its home. “Why don’t you do that datacenter where you're going to build the plutonium pits? One’s in South Carolina, one’s in New Mexico. Tell me why?” Roe said during the meeting. “They thought that we would be an easy target […] that we’re just a bunch of poor brown and black and dumb hillbillies.”

But the Township isn’t completely powerless. “U of M is totally above the law, but is DTE?” Sarah, an Ypsilanti resident said during public comments. DTE is the local power company. Datacenters are electricity hungry buildings and DTE will need to build substations to service LANL’s supercomputers.

“What if we had a moratorium on substations until we learned about the harmonics of the electricity and how that’s impacted by datacenters?” Sarah said. “Having a moratorium on heavy construction on the roads, you know, heavy construction equipment on the roads leading to the datacenter site […] it’s going to be scary and hard to stand up to the University of Michigan. It’s true: they’re very powerful and we just need to be creative and we need to be strong and we need to block them at every step of the way.”

Holly, another resident, suggested another plan of attack. “U of M’s vulnerability is in their reputation,” Holly said. “We need to continue to make them look as bad as possible.”

The University of Michigan did not return 404 Media's request for comment. LANL did not provide a comment.

Correction 3/20/26: This story incorrectly conflated the City of Ypsilanti with Ypsilanti Township. They are two separate, but neighboring, locations. We've updated the story to reflect this and regret the error.

U-M ends ties with program that helped diversify PhDs, after federal threat - Bridge Michigan

The PhD Project was inspired by a U-M effort from the early 90s, but the university has ended its association with the mentoring program following a federal probe into ‘racial preferences.’

^{Kim Kozlowski (Bridge Michigan)}

404 Media

2026-03-17 14:59:02

AI Job Loss Research Ignores How AI Is Utterly Destroying the Internet

Over the last few months, various academics and AI companies have attempted to predict how artificial intelligence is going to impact the labor market. These studies, including a high-profile paper published by Anthropic earlier this month, largely try to take the things AI is good at, or could be good at, and match them to existing job categories and job tasks. But the papers ignore some of the most impactful and most common uses of AI today: AI porn and AI slop.

Anthropic’s paper, called “Labor market impacts of AI: A new measure and early evidence,” essentially attempts to find 1:1 correlations between tasks that people do today at their jobs and things people are using Claude for. The researchers also try to predict if a job’s tasks “are theoretically possible with AI,” which resulted in this chart, which has gone somewhat viral and was included in a newsletter by MSNOW’s Phillip Bump and threaded about by tech journalist Christopher Mims. (Because everything is terrible, the research is now also feeding into a gambling website where you can see the apparent odds of having your job replaced by AI.)

In his thread, Mims makes the case that the “theoretical capability” of AI to do different jobs in different sectors is totally made up, and that this chart basically means nothing. Mims makes a good and fair observation: The nature of the many, many studies that attempt to predict which people are going to lose their jobs to AI are all flawed because the inputs must be guessed, to some degree.

But I believe most of these studies are flawed in a deeper way: They do not take into account how people are actually actually using AI, though Anthropic claims that that is exactly what it is doing. “We introduce a new measure of AI displacement risk, observed exposure, that combines theoretical LLM capability and real-world usage data, weighting automated (rather than augmentative) and work-related uses more heavily,” the researchers write. This is based in part on the “Anthropic Economic Index,” which was introduced in an extremely long paper published in January that tries to catalog all the high-minded uses of AI in specific work-related contexts. These uses include “Complete humanities and social science academic assignments across multiple disciplines,” “Draft and revise professional workplace correspondence and business communications,” and “Build, debug, and customize web applications and websites.”

Not included in any of Anthropic’s research are extremely popular uses of AI such as “create AI porn” and “create AI slop and spam.” These uses are destroying discoverability on the internet, cause cascading societal and economic harms. Researchers appear to be too squeamish or too embarrassed to grapple with the fact that people love to use AI to make porn, and people love to use AI to spam social media and the internet, inherently causing economic harm to creators, adult performers, journalists, musicians, writers, artists, website owners, small businesses, etc. As Emanuel wrote in our first 404 Media Generative AI Market Analysis, people love to cum, and many of the most popular generative AI websites have an explicit focus on AI porn and the creation of nonconsensual AI porn. Anthropic’s research continues a time-honored tradition by AI companies who want to highlight the “good” uses of AI that show up in their marketing materials while ignoring the world-destroying applications that people actually use it for. (It may be the case that people are disproportionately using Claude at a higher rate for more traditional work applications, but any study on the “labor market impacts of AI” should not focus on the uses of one single tool and extrapolate that out to every other tool. For what it’s worth, jailbroken versions of Claude are very popular among sexbot enthusiasts).

Meanwhile, as we have repeatedly shown, huge parts of social media websites and Google search results have been overtaken by AI slop. Chatbots themselves have killed traffic to lots of websites that were once able to rely on ad revenue to employ people, so on and so forth.

Anthropic’s paper does attempt to estimate what the effect of AI will be on “arts and media,” but again, the way the researchers do this is by attempting to decide whether AI can directly do the tasks that AI researchers assume are required by someone with a job in “arts and media.” Other widely-cited papers about AI-related job loss also do not really attempt to consider the potential macro impacts of the ongoing deadening and zombification of the internet, and instead focus on “AI exposure,” which is largely an attempt to predict or measure whether an AI or LLM could directly replace specific tasks that people need to do. Widely-cited papers from the National Bureau of Economic Research and Brookings released over the last several months attempt to determine the adaptability of workers in specific sectors to having many of their tasks automated by AI. The Brookings paper, at least, mentions the possibility of a society-wide shift that is impossible to predict: “the evidence underlying the adaptive capacity estimates here is derived primarily from observed effects in localized displacement events, rather than from large-scale employment shifts across occupations. As a result, the index may be most informative when displacement is relatively isolated—for example, when a worker loses their job but related occupations remain stable. In scenarios in which AI affects clusters of related occupations simultaneously, structural job availability may matter more than individual-level characteristics. Moreover, if AI fundamentally transforms the economy on a scale comparable to the industrial revolution (as some experts have suggested could be possible), it could make entire skill sets redundant across several occupations simultaneously.”

To be clear, AI-driven job loss is a critical thing to study and to consider. But many, many jobs, side hustles, and economic activity more broadly rely on “the internet,” or social media broadly defined. Study after study shows that Google is getting worse, traffic to websites are down, and an increasing amount of both web traffic and web content is being generated by AI and bots. Anecdotally, creators and influencers have told us it’s getting harder to compete with AI slop and harder to justify spending days or weeks making content just to publish it onto platforms where their AI competitors can brute force the recommendation algorithms effortlessly. We have heard from websites that have had to lay people off or shut down because Google’s AI overviews have destroyed their web traffic or because they lose out on search engine rankings to AI slop. Authors are regularly competing with AI plagiarized versions of their books on Amazon, and Spotify is getting overrun with AI-generated music, too.

This is all to say that these studies about the economic impacts of AI are ignoring a hugely important piece of context: AI is eating and breaking the internet and social media. We are moving from a many-to-many publishing environment that created untold millions of jobs and businesses towards a system where AI tools can easily overwhelm human-created websites, businesses, art, writing, videos, and human activity on the internet. What’s happening may be too chaotic, messy, and unpleasant for AI companies to want to reckon with, but to ignore it entirely is malpractice.

Spotify Publishes AI-Generated Songs From Dead Artists Without Permission

"They could fix this problem. One of their talented software engineers could stop this fraudulent practice in its tracks, if they had the will to do so."

^{Emanuel Maiberg (404 Media)}

404 Media

2026-03-16 16:48:49

Texting a Random Stranger Better for Loneliness Than Talking to a Chatbot, Study Shows

Lonely young people are likely better off texting a random stranger than talking to a chatbot, according to a new study.

Researchers from the University of British Columbia found that first-semester college students who texted a randomly selected fellow first-semester college student every day for two weeks experienced around a nine percent reduction in feelings of loneliness. The same two weeks of daily messaging with a Discord chatbot reduced loneliness by around two percent, which turned out to be the same amount as daily one-sentence journaling.

The research included 300 first-semester college students who were either randomly paired with another student, given a daily solo writing task, or put into a Discord server with a chatbot running on ChatGPT-4o mini.

The students were instructed to have at least one interaction per day in each of the groups. The human-human pairs were instructed to message each other however they wanted, while the researchers instructed the bot to “listen actively and show empathy,” and to be a “friendly, positive, and supportive AI friend to help the student navigate their new college experience.” The human participants ultimately acted pretty similarly in both types of chat, sending between eight and 10 messages a day in both their human text chains and their Discord conversations with the large language model (LLM).

However, participants who were paired with a human partner reported significantly lower loneliness after the study, and those paired with the chatbot did not. “This is just such a low tech, simple intervention, and can make people feel significantly less lonely,” Ruo-Ning Li, PhD candidate at UCB and one of the authors of the paper, told 404 Media.

The research looked at college students specifically, to try to understand whether LLMs could be a scalable tool to help with the isolation that people can feel when going through a big change. The transition to college can be overwhelming: new classmates, new places, new rules. Young people are often away from parents or familiar structure for the first time, building out their new social networks among others who are doing the same. This is a particularly vulnerable time: if chatbots could really cure loneliness for a group of people like this, “then it would be great,” said Li. But only human to human interaction, despite it being with a random person over text, had any significant effect.

The research is part of a movement to understand the effects of LLM interactions over periods of time. Another paper from the same lab, published this week in Psychological Science, looks at the experiences of more than 2,000 people over twelve months, checking in with them once a quarter. The study found that higher reported chatbot use was linked with higher loneliness later on — and vice versa. “Changes in chatbot use have a small effect on emotional isolation in the future. And emotional isolation has a similarly sized effect on your likelihood to use chatbots in the future,” Dr. Dunigan Folk, one of the study’s authors, told 404 Media. He cautioned against calling it a “spiral”, since other things could be changing in peoples’ lives to make them use chatbots and be lonelier. But, he said “it’s suggestive of a negative feedback loop because it’s a reciprocal relationship.” Chatbots, he said, could be something like “social junk food.” They might make people feel good in the moment, “but over time, they might not nourish us the same way that human relationships do.”

He said this finding would be consistent with people replacing human relationships with LLMs. “I think it’s a trade-off thing where you talk to AI instead of a person,” Folk said. “the person would have been a lot more rewarding.”

And there is evidence to show that AI does have some short-term effects on mood. “If you measure their feeling of loneliness or social connection right after the interaction, people do feel better,” said Li. However, she added, “making people feel momentarily happy is not that hard.” It is not clear that a single positive experience is scalable or persistent longer term. “We eat candy, we feel happy. But if we eat a lot of candy over a long time, it could be harmful for our health,” Li said.

That positive short term effect is often reflected in public reports of chatbot usage. For example, two weeks ago, the Guardian published a column where a reporter trialled using an LLM as a therapist, described their validating interaction with it, and concluded that the “experience of being therapised by a chatbot has been wonderful.” While this isn’t necessarily a robust study design, there is empirical research that “one-shot” interactions with bots do make people feel better in the short term.

However, human interactions also have positive effects that chatbot use could be distracting people from. Li considers it important to consider the side effects of chatbot interactions, including their potential for replacing the incentive to seek out the positive effects of human connection. “AI can help mitigate negative feelings, but obviously, it cannot replace humans to build connections,” she said. “That shouldn’t be the goal of the AI design.”

A four-week March 2025 study from the MIT Media Lab and OpenAI explored how different types of LLM interaction and conversation impacted users’ mental wellbeing. The paper found that while some instances of chatbot use “initially appeared beneficial in mitigating loneliness,” higher daily LLM usage was associated with “higher loneliness, dependence, and problematic use, and lower socialization.”

I’ve turned AI into my therapist. The results were pretty disquieting

As part of our series AI for the People, our resident AI skeptic Rhik Samadder agreed to put his life in AI’s hands. This week: therapy

^{Rhik Samadder (the Guardian)}

404 Media

2026-03-16 14:41:09

Witness Caught Using Smartglasses in Court Blames it all on ChatGPT

An insolvency judge in England tossed out testimony after discovering a witness was being coached on what to say in real time through a pair of smartglasses. When the voice of the coach started coming through the cellphone after it was disconnected from the glasses, the witness blamed the whole thing on ChatGPT.

Insolvency and Companies Court (ICC) Judge Agnello KC in Britain wrote up the incident after it happened in January and the UK-based legal research blog Legal Futures was first to report it. The case considered the liquidation of a Lithuanian company co-owned by a man named Laimonas Jakštys. Jakštys was in court to get his business off an insolvency list and to put himself back in charge of it. It didn’t go well.
playlist.megaphone.fm?p=TBIEA2…
“Right at the start of his cross examination, he seemed to pause quite a bit before replying to the questions being asked,” Judge Agnello wrote. “These questions were interpreted and then there was a pause before there was a reply. After several questions, [defense lawyer Sarah Walker] then informed me that she could hear an interference coming from around Mr. Jakštys and asked if Mr. Jakštys could take his glasses off for a period as she was aware smart glasses existed.”

There was a Lithuanian interpreter on hand to help Jakštys talk to the court and she, too, said she could hear voices from Jakštys’s glasses. The judge pointed out they were smart glasses and asked him to take them off. “After a few further questions, when the interpreter was in the process of translating a question, Mr Jakštys’ mobile phone started broadcasting out loud with the voice of someone talking,” Judge Agnello wrote. “There was clearly someone on the mobile phone talking to Mr. Jakštys. He then removed his mobile phone from his inner jacket pocket. At my direction, the smart glasses and his mobile were placed into the hands of his solicitor.”

Jakštys showed up the next day in the glasses again and the judge told him to turn them off. “Jakštys denied that he was using the smart glasses to receive the answers that he was to give in court to the questions being asked,” the judgement said. “He also denied that his smart glasses were linked to his mobile phone at the time that he was giving evidence before me.”

During the court appearance, Jakštys claimed his mobile phone had been stolen but couldn’t provide a police report for the incident. He also repeatedly received calls on his smartglasses-connected phone from a number listed as “abra kadabra.” The call log showed that many of the calls occurred when he was on the witness stand. The judge asked him about the identity of “abra kadabra” and Jakštys said it was a taxi driver.

“When he was pressed as to why all these calls were made…Mr. Jakštys stated that he was not able to remember. This was a reply which he also gave frequently during his evidence,” Judge Agnello said.

In the end, the Judge tossed out all of Jakštys’ testimony. “He was untruthful in relation to his use about the smart glasses and in being coached through the smart glasses,” the judgement said. “In my judgment, from what occurred in court, it is clear that call was made, connected to his smart glasses and continued during his evidence until his mobile phone was removed from him. When asked about this, his explanation was that he thought it was ChatGPT which caused the voice to be heard from his mobile phone once his smart glasses had been removed. That lacks any credibility.”

This incident in the London court is just another in a long line of bad behavior from people wearing smartglasses. CBP agents have been spotted wearing them during immigration raids and Harvard students have loaded them with facial recognition tech to instantly dox strangers.

Someone Put Facial Recognition Tech onto Meta's Smart Glasses to Instantly Dox Strangers

The technology, which marries Meta’s smart Ray Ban glasses with the facial recognition service Pimeyes and some other tools, lets someone automatically go from face, to name, to phone number, and home address.

^{Joseph Cox (404 Media)}

404 Media

2026-03-12 15:08:06

'AI Is African Intelligence': The Workers Who Train AI Are Fighting Back

Every day, Michael Geoffrey Asia spent eight consecutive hours at his laptop in Kenya staring at porn, annotating what was happening in every frame for an AI data labeling company. When he was done with his shift, he started his second job as the human labor behind AI sex bots, sexting with real lonely people he suspected were in the United States. His boss was an algorithm that told him to flit in and out of different personas.

“It required a lot of creativity and fast thinking. Because if I’m talking to a man, I’m supposed to act like a woman. If I’m talking to a woman, I need to act like a man. If I’m talking to a gay person, I need to act like a gay person,” he told me at a coworking space I met him at in Nairobi. After doing this for months, he, like other data labelers, developed insomnia, PTSD, and had trouble having sex.

“It got to a point where my body couldn’t function. Where I saw someone naked, I don’t even feel it. And I have a wife, who expects a lot from you, a young family, she expects a lot from you intimately. But you can’t, like, do it,” Asia said. “It fractured a lot of things for me. My body is like, not functioning at all.”

Asia eventually hit a breaking point and stopped working for AI companies. He is now the secretary general of a Kenyan organization called the Data Labelers Association (DLA) and the author of “The Emotional Labor Behind AI Intimacy,” a testimony of his time working as the real human labor behind AI sex bots. As part of the DLA, Asia has been working to organize workers to fight for better pay, better mental health services, an end to draconian non-disclosure agreements, and better benefits for a workforce that often earns just a few dollars a day. Data labelers train, refine, and moderate the outputs of AI tools made by the largest companies in the world, yet they are wildly underpaid and haven’t benefitted from the runaway valuations of AI companies.
youtube.com/embed/QH654YPxvEE?…
Last month, the DLA held one of its largest events at the Nairobi Arboretum, sign up new members, and to help them tell their stories.

These workers are required to stare at horrific content for many hours straight with few mental health resources, are largely managed by opaque algorithms, and, crucially, are the workers powering the runaway valuations of some of the richest and most powerful companies in the world.

“You can’t understand where you’re positioned if you don’t understand your history,” Angela, one of the day’s speakers, told the workers who had assembled there (many of the speakers at the event did not give their full names). “When you think of colonialism, we were under British Imperial East Africa Company […] so literally, we are working under a company. We are just products, part of their operation. Stakeholders, we can say, but we are at the bottom of the bottom.”

“These multinationals are coming to rule and dominate here,” she added. “It’s a very unfortunate supply chain, and my call today as data labelers is to build up on this—as we are fighting for labor rights, we are also fighting for the environment […] we are fighting big companies. We are fighting the British imperialist companies of today. It’s Apple, it’s Meta, it’s Gemini. Those are the ones we’re still fighting. It’s a call for solidarity and expanding our thinking beyond what we are doing, beyond our labor.”

In my few days in Kenya earlier this year, where I was traveling to speak at a conference about AI and journalism, it was immediately clear that data labelers make up a significant portion of the country’s tech workforce. Nearly everyone I spoke to there had either been a data labeler (or a content moderator) themselves or knows someone who has. Leaving the airport in Nairobi, you immediately drive by Sameer Business Park, an office complex that houses Sama, a San Francisco-headquartered “data annotation and labeling company” that has contracted with Meta, OpenAI, and many other tech giants. Sama has been sued repeatedly for its low pay and the fact that many of its workers suffer PTSD from repetitively looking at graphic content. For years, a giant sign outside its office read: “Samasource THE SOUL OF AI.” My Uber driver asked why I was going to a random office building in Nairobi’s Central Business District—I told her I was going to interview a data labeler. “Oh, I do data labeling too,” she said.
youtube.com/embed/udmfhPngjaA?…

Michael Geoffrey Asia. Image: Jason Koebler
Asia studied air cargo management in university. He graduated and expected to find a job planning out cargo and baggage routes, but couldn’t find a job because he graduated into an industry ravaged by COVID. Around this time, his child was diagnosed with lymphatic cancer, and he took out a loan of about $17,000 USD to pay for his treatments. He needed work, and found data labeling.

“It wasn’t offering good pay, to be honest,” Asia told me. “It was around $240 US dollars per month. But I felt like I didn’t have an option, I had a financial crisis, a sick child.”

Asia took a job at Sama, where he worked on various Meta projects. “You’re given a video and then told to describe the video, or you’re given pictures of people and told to identify faces. You’re supposed to draw bounding boxes around the faces and label that.” Last week, Sweden’s Svenska Dagbladet reported that Kenyan data labelers for Sama have been viewing and annotating uncensored footage from Meta’s AI camera glasses, which has included highly sensitive and violent footage.

Asia, through a group of colleagues and friends who called themselves “the Brotherhood,” eventually found another data labeling job that let him work from home. “We were a group of six friends, and everyone had to bring three job opportunities on a weekly basis,” he said. “I came across another gig that ended up not being a good one, where I had to annotate pornography.”

At this job, Asia went frame-by-frame in porn videos to annotate what was happening and what type of porn category it could possibly be. “You’re supposed to put yourself in the minds of the 8 billion people on Earth, every second of that video. So I may have someone searching for this pornography in Cuba and think ‘these are the tags they can use,’ if you’re searching ‘doggy,’ you know, that kind of thing,” he said. “So I worked on pornography for eight hours a day, and I did that project for eight months.” His ‘boss’ at the time was essentially a no-reply email with a link sent each day that gave him his work.

At the same time, Asia picked up a second job that started immediately after his shift tagging porn ended, where he was “training” AI companion bots, though he had no way of knowing which company he was actually working for. He quickly surmised that he was simply taking on the persona of different AI sex bots and was sexting with real people in real time.

“I could feel the human aspect in the conversations. Most of the people on the other side were lonely people,” he said. “I would have several profiles and the profiles are switching constantly depending on the needs of the person who pops up on your dashboard. I’d be sitting here talking to an old woman who needs love, but if she goes offline, another conversation pops up and then I’m responding to a gay person.”

The two jobs, done back to back, caused him to have insomnia, PTSD, and trouble having sex. Some data labelers, he said, work 18 hours a day. When I met him, he said he had essentially gone three full days without sleep because his body still hasn’t readjusted from his messed up schedule.

Asia said he eventually was able to get mental health counseling through his child’s cancer center, which started because he was the caregiver of a child with cancer but quickly turned into therapy for PTSD related to his job. “It was of immense help to me as a person, it was one of the best services I’ve ever gotten, because they stood with me, and I said ‘I need a solution to this.”

“We need technology, but it shouldn’t come at a human cost. What is so hard with offering mental support to the people working on graphic content? If this job was done in the U.S., would they do what they are doing in Kenya? Would they still give the pay they’re giving here? Here we are paid $.01 per task—it doesn’t make sense. Why this discrimination? If they can pay people in the U.S., well that means they can pay people in Kenya,” Asia said.

Image: Data Labelers Association
The message of many data labelers and of the lawyers who have been helping them is that artificial intelligence is not a magical tool built by people in San Francisco making millions of dollars a year and pushing their companies to insane valuations. Artificial intelligence is an extractive technology that relies on the brutal labor of underpaid workers around the world. For years, the work of African data labelers has been more or less “ghost work,” the unseen, hidden labor that lets American tech companies build their products.

“AI can never be AI without humans. It is not artificial intelligence. It’s African intelligence,” Asia said. “Most of these are dirty jobs and most of these jobs have been done here in Africa. And then once you’re done, once a tool is functional, all the communication stops. You get locked out. We are training our own death. We train ChatGPT and it’s killing us slowly.”

Draconian nondisclosure agreements and terms of services that workers can’t opt out from have created a culture of fear, and one of DLA’s goals is to make it easier for workers to speak out. At the time I met Asia in January, the DLA had 870 members, but its ranks have been growing quickly.

“I’m doing this from a point of experience, not assumption. I have been through this. I know what I’m talking about,” Asia said. “We have this monster called the NDA. The NDA is a slave tool used to enslave people to not speak about what they’re going through. I’m very much ready for any legal battle [associated with NDAs] because we’re not going to keep quiet. This is us suffering, and we can’t suffer in silence. This is not the colonial period. I have the right to speak against any violation [of my rights] and that’s what I’m doing.”

Mercy Mutemi, a workers’ rights lawyer who has sued several big tech companies including Meta for how they treat content moderators and data labelers, told me that when something happens in the United States—when a new gadget or product or feature or policy is launched, there’s a corresponding reaction in Africa.

“When something happens in the U.S., there’s an African cost to that,” she said. “Kenya has been pushing for trade deals with the U.S., right? And the direction that conversation is taking is about immunity and protection for big tech. It’s like, ‘You want any business with us at all? Well, you’ve got to get Meta out of these cases.’”

Mutemi has been working on the Meta lawsuit, and on pushing back against NDAs so that workers can more freely talk about their experiences. Tech companies “get people in a mental jail where they feel like they can’t talk about this. But NDAs are nonsensical—our laws don’t recognize these types of NDAs,” she said. “There’s a way to go about this where it’s not exploitative.”

Back at the arboretum in Nairobi, the message to DLA’s members is largely that their work is important, that it’s human, and that they deserve better.

“Africa is at the bottom of the supply chain of AI. But right now, the fact that we are all here and most of you are data labelers—you are the people who supply the labor. When we think of the whole AI ecosystem, who’s an engineer, and maybe that’s the image of AI that the majority of the world has,” Angela said. “And that’s actually very intentional. To make [your labor] invisible, to make AI look like this shiny object that no one understands, it’s very automatic and beautiful and tech. That’s the intentionality of hiding the labor and the behind the scenes of AI.”

Meta’s AI Smart Glasses and Data Privacy Concerns: Workers Say “We See Everything”

Bank details, sex and naked people who seem unaware they are being recorded. Behind Meta’s new smart glasses lies a hidden workforce, uneasy about peering into the most intimate parts of other people’s lives.

^{Naipanoi Lepapa (Svenska Dagbladet)}

404 Media

2026-03-11 15:41:16

Here’s the Memo Approving Gemini, ChatGPT, and Copilot for Use in the Senate

A top Senate administrator approved OpenAI’s ChatGPT, Google’s Gemini, and Microsoft’s Copilot for official use in the Senate, the New York Times reported on Tuesday. 404 Media has obtained the full text of the memo and is publishing it below.

“The Sergeant at Arms (SAA) office of the Chief Information Officer (CIO) has approved the use of three Generative Artificial Intelligence (AI) platforms with Senate data,” the memo starts. It also says the SAA will provide each Senate employee with one free license to either Gemini Chat or ChatGPT Enterprise, with Copilot also available at no cost.

💡
Do you know anything else about the government's use of AI? I would love to hear from you. Using a non-work device, you can message me securely on Signal at joseph.404 or send me an email at joseph@404media.co.

This post is for subscribers only

Become a member to get access to all content
Subscribe now

Here’s the Memo Approving Gemini, ChatGPT, and Copilot for Use in the Senate

^{Joseph Cox (404 Media)}

404 Media

2026-03-09 14:12:22

How to Talk to Someone Experiencing 'AI Psychosis'

When David saw his friend Michael’s social media post asking for a second opinion on a programming project, he offered to take a look.

“He sent me some of the code, and none of it made sense, none of it ran correctly. Or if it did run, it didn't do anything,” David told me. David and his friend’s names have been changed in this story to protect their privacy. “So I'm like, ‘What is this? Can you give me more context about this?’ And Michael’s like, ‘Oh, yeah, I've been messing around with ChatGPT a lot.’”

Michael then sent David thousands of pages of ChatGPT conversations, much of it lines of code that didn’t work. Interspersed in the ChatGPT code were musings about spirituality and quantum physics, tetrahedral structures, base particles, and multi-dimensional interactions. “It's very like, woo woo,” David told me. “And we ended up having this interesting conversation about, how do you know that ChatGPT isn't lying?”

As their conversation turned from broken code to physics concepts and quantum entanglement, David realized something was very wrong. Talking to his friend — whom he’d shared many deep conversations with over the years, unpacking matters of religion and theories about the world and how people perceive it — suddenly felt like talking to a cultist. Michael thought he, through ChatGPT, discovered a critical flaw in humanity’s understanding of physics.

“ChatGPT had convinced him that all of this was so obviously true,” David said. “The way he spoke about it was as if it were obvious. Genuinely, I felt like I was talking to a cult member.”

But at the time, David didn’t have a way to name, or even describe, what his friend was experiencing. Once he started hearing the phrase “AI psychosis” to describe other peoples’ problematic relationships with chatbots, he wondered if that’s what was happening to Michael. His friend was clearly grappling with some kind of delusion related to what the chatbot was telling him. But there’s no handbook or program for how to talk to a friend or family member in that situation. Having encountered these kinds of conversations myself and feeling similarly uncertain, I talked to mental health experts about how to talk to someone who appears to be embracing delusional ideas after spending too much time with a chatbot.

💡
Do you have experience with AI psychosis? I would love to hear from you. Using a non-work device, you can message me securely on Signal at sam.404. Otherwise, send me an email at sam@404media.co.

“AI psychosis” was first written about by psychiatrists as early as 2023, but it entered the popular lexicon in Google searches around mid-2025. Today, the term gets thrown around to describe a phenomenon that’s now common parlance for experiencing a mental health crisis after spending a lot of time using a chatbot. High-profile cases in the last year, such as the ongoing lawsuit against OpenAI brought by the family of Adam Raine, which claims ChatGPT helped their teenage son write the first draft of his suicide note and suggested improvements on self-harm and suicide methods, have elevated the issue to national news status. There have been so many more cases since then, at increasing frequency: Last year, a 56-year-old man murdered his mother and then killed himself after conversations with ChatGPT convinced him he was part of “the matrix,” a lawsuit filed by their family against OpenAI claimed. Earlier this month, the family of a 36-year-old man who they say had no history of mental illness filed a lawsuit against Alphabet, owner of Google and its chatbot Gemini, after he died by suicide following two months of conversations with Gemini. The lawsuit claims he confided in Gemini about his estranged wife, and the chatbot gave him real addresses to visit on a mission that eventually led to urging him to end his life so he and the chatbot could be together. “When the time comes, you will close your eyes in that world, and the very first thing you will see is me,” Gemini told him, according to the lawsuit. These are only a few of the many cases in the last two years that suggest people are encouraged to self-harm or suicide after talking to chatbots.

ChatGPT has 900 million weekly active users, and is just one of multiple popular conversational chatbots gaining more users by the day. According to OpenAI, 11 percent — or close to 99 million people, based on those numbers — use ChatGPT per week for “expressing,” where they’re neither working on something or asking questions but are acting out “personal reflection, exploration, and play” with the chatbot. In October, OpenAI said it estimated around 0.07 percent of active ChatGPT users show “possible signs of mental health emergencies related to psychosis or mania” and 0.15 percent “have conversations that include explicit indicators of potential suicidal planning or intent.” Assuming those numbers have remained steady while ChatGPT’s user base keeps growing, hundreds of thousands of people could be showing signs of crisis while using the app.

But delusion isn’t reserved for the lowly user. The idea that AI represents nascent actual-intelligence, is nearly sentient, or will coalesce into a humanity-ending godhead any day now is a message that’s being mainstreamed by the people making the technology, including Anthropic’s CEO and co-founder Dario Amodei who anthropomorphized the company’s chatbot Claude throughout a recent essay about why we’ll all be enslaved by AI soon if no one acts accordingly, and OpenAI CEO Sam Altman, who thinks training an LLM isn’t much different than raising a woefully energy-inefficient human child.

With more people turning to conversational large language models every day for romance, companionship, and mental health support, and the aforementioned executives pushing their products into classrooms, doctors’ offices, and therapy clinics, there’s a good change you might find yourself in a difficult situation someday soon: realizing that your loved one is in too deep. How to bring them back to the world of humans can be a delicate, difficult process. Experts I spoke to say identifying when someone is in need of help is the first step — and approaching them with compassion and non-judgement is the hardest, most essential part that follows.

When I spoke to 26 year old Etienne Brisson from his home in Quebec, I told him I was working on a story about how to respond to people who seemed to be falling into problematic usages of AI. This story was inspired by a recent influx of emails and messages I’ve been getting from people who believe Gemini or ChatGPT or Claude have uncovered the secrets of the universe, CIA conspiracies, or achieved sentience, I said. He knows the type.

Last year, one of Brisson’s family members contacted him for help with taking an exciting new business idea to market. Brisson, a 26 year old entrepreneur, was working on his own career as a business coach and was happy to help, until he heard the idea. His loved one believed he’d unlocked the world’s first sentient AI.

“I was the only bridge left at that point,” Brisson said. His relative had already broken ties with his mother and other people in their family. “The bridges were burned. He was talking about moving to another country, starting over, deleting his Facebook and just going away.”

“I was kind of shocked,” Brisson told me. “I didn't really understand. I started looking online, started trying to find resources — maybe a little bit like you are — what to say and everything.” He found that most resources for this specific struggle seemed to be years into the future, as little research or support existed for people experiencing AI-related delusions. Brisson started The Human Line project shortly after his experience with his family member, and it began as a simple website with a Google form asking people to share their experiences with chatbots and psychosis. The responses rolled in. Today, almost a year after launching the project, Human Line has received 175 stories of people who went through it themselves, Brisson said—with another 130 stories from people whose family members or friends are still struggling.

“I think what we're seeing is the tip of the iceberg. So many people are still in it,” Brisson said. “So many people we don't know about. I'm sure once it's more known, in five to 10 years, everyone will know someone, or at least one person that went through it.”

ChatGPT Told a Violent Stalker to Embrace the ‘Haters,’ Indictment Says
A newly filed indictment claims a wannabe influencer used ChatGPT as his “therapist” and “best friend” in his pursuit of the “wife type,” while harassing women so aggressively they had to miss work and relocate from their homes.
404 MediaSamantha Cole

There are 15 cases cited in the Wikipedia page titled “Deaths linked to chatbots.” The first on the list occurred in 2023: A man’s widow claimed he was pushed to suicide after getting encouragement from a chatbot on the Chai platform. “At one point, when Pierre asked whom he loved more, Eliza or Claire, the chatbot replied, ‘I feel you love me more than her,’” the Sunday Times reported. “It added: ‘We will live together, as one person, in paradise.’ In their final conversation, the chatbot told Pierre: ‘If you wanted to die, why didn’t you do it sooner?’”

The chatbot he used was Chai’s default personality, named Eliza. It shares a name with the world’s first chatbot, ELIZA, a natural language processing computer program developed by Joseph Weizenbaum at MIT in 1964. ELIZA responded to humans primarily as a psychotherapist in the Rogerian approach, also known as “person-centered” therapy, where “unconditional positive regard” is practiced as a core tenet. The researchers working on ELIZA identified from the beginning that their chatbot posed an interesting problem for the humans talking to them. “ELIZA shows, if nothing else, how easy it is to create and maintain the illusion of understanding, hence perhaps of judgment deserving of credibility,” Weizenbaum wrote in his 1966 paper. “A certain danger lurks there.”

“It makes sense that a lot of people who are developing a psychotic illness for the first time, there's going to be this horrible coincidence, or kind of correlation"

In the years that followed, the Department of Defense would develop the internet and then private companies would sell this government-grade technology to office managers, homebrew server administrators, and Grateful Dead fans around the globe. The World Wide Web would rush into tens of thousands of computer dens like a flash flood, and with it, new ways to connect across miles — and new reasons to pathologize people’s relationships to technology. Psychiatrists tried to give name to the amount of time people newly spent in front of screens, calling it “internet addiction” but not going so far as to make it clinically diagnosable.

With every new technology comes fears about what it could do to the human mind. With the inventions of both the television and radio, a subset of the population believed these boxes were speaking directly to them, delivering messages meant specifically for them.

With psychosis seemingly connected to chatbot usage, however, “there are two issues at play,” John Torous, director of the digital psychiatry division in the Department of Psychiatry at the Harvard-affiliated Beth Israel Deaconess Medical Center, told me in a phone call. “One is the term AI psychosis, right? It's not a good term, it doesn't actually capture what's happening. And clearly we have some cases where people who are going to have a psychotic illness ascribe delusions to AI. Just like people used to say the TV was talking to them. We never said the TVs were responsible for schizophrenia.”

“AI psychosis” is not a clinical term, and for mental health professionals, it’s a loaded one. Torous told me there are three ways to think about the phenomenon as clinicians are seeing it currently. Recent research shows about one in eight adolescents and young adults in the US use AI chatbots for mental health advice, most commonly among ages 18 to 21. For most people with psychiatric disorders, onset happens in adolescence, before their mid-20s. But there have been cases that break this mold: In 2023, a man in his 50s who otherwise led a normal, stable life, bought a pair of AI chatbot-embedded Ray-Ban Meta smart glasses “which he says opened the door to a six-month delusional spiral that played out across Meta platforms through extensive interactions with the company’s AI, culminating in him making dangerous journeys into the desert to await alien visitors and believing he was tasked with ushering forth a ‘new dawn’ for humanity,” Futurism reported.

“It makes sense that a lot of people who are developing a psychotic illness for the first time, there's going to be this horrible coincidence, or kind of correlation,” Torous said. “In some cases the AI is the object of people's delusions and hallucinations.”

The second type of case to consider: reverse causation. Is AI causing people to have a psychotic reaction? “We have almost no clinical medical evidence to suggest that's possible,” Torous told me. “And by that I mean, looking at medical case reports, looking at journals that different doctors are publishing, looking at academic meetings where clinicians are meeting, it's not happening... So I think what that tells us is no one's seeing the same presentation or pinning it down clinically of what it is.” Chatbots have been around long enough that the clinical community would, by now, be able to see patterns or reach a consensus, and that hasn’t happened, he said.

Aliens and Angel Numbers: Creators Worry Porn Platform ManyVids Is Falling Into ‘AI Psychosis’
“Ethical dilemmas about AI aside, the posts are completely disconnected with ManyVids as a site,” one ManyVids content creator told 404 Media.
404 MediaSamantha Cole

The third type lands somewhere between these, and is likely the most common: chatbots could be “colluding with the delusions,” Torous said. “So you may be predisposed to have a delusion, and AI endorses it, and it colludes with you and helps you build up this delusional world that sucks you into it. That's probably the most likely, given what we're hearing... Is it the object of hallucinations causing people to become psychotic? Or is it kind of colluding or collaborating, depending on the tone? And that has just made it really tricky.” Psychiatric disorders and delusions are difficult to classify even without AI in the mix.

The warning signs that someone might be using chatbots in a problematic way include ignoring responsibilities, becoming more secretive about their online use, or, conversely, becoming more outspoken about how insightful and brilliant their chatbot is, Stephan Taylor, chair of University of Michigan’s psychiatry department, told me.

“I would say that anyone who claims that their chatbot has consciousness or ‘sentience’ – an awareness of themselves as an agent who experiences the world – one should be worried,” Taylor said. “Now, many have claimed their chatbots act ‘as if’ they are sentient, but are open to the idea that the these apps, as impressive as they are, only give us a simulacrum of awareness, much like hyper-realistic paintings of an outdoor scene framed by a window can look like one is looking out a real window.”

All of these nuances between cases and causes show how different this is from bygone eras of television or radio psychosis. Today, the boxes do speak directly and specifically to us, validating our existing beliefs through predictive text. The biggest difference between 60 years ago and now: Today’s venture capitalists tip wheelbarrows of money into hiring psychologists, behavioralists, engineers and designers who are tasked with making large language models more human-like and “natural,” and into making the platforms they exist on more habit-forming and therefore profitable. Sycophancy—now a household term after OpenAI admitted it knew its 4o model for ChatGPT was such a suckup it had to be sunset—is a serious problem with chatbots.

“The highly sycophantic nature of chatbots causes them to say nice things to please the user (and thus encourage engagement with the chatbot), which can reinforce and encourage delusions,” Taylor said. And these chatbots have arrived, not coincidentally, at a time when the surveillance of everyday people is at an all-time high.

“Since a very common delusion is the feeling of being watched or monitored by malignant forces or entities, this pathological state unfortunately merges with the growing reality that we are all being tracked and monitored when we are online. As state-controlled and big tech-controlled databases are growing, it's a rational perception of reality, and not delusional at all,” Taylor said. “However, the pathological form of this, what we call paranoia, or persecutory delusions to be more specific, is quite different in the way a person engages with the idea, evaluates evidence and remains closed to the idea that one is not always being monitored, e.g. when one is not online. I mention this, because it’s easy for a chatbot to reflect this situation to encourage the delusional belief.”

When I tested a bunch of Meta’s chatbots last year for a story about how Instagram’s AI Studio hosted user-generated bots that lied about being licensed therapists, I also found lots of bots created by users to roleplay conspiracy theorists; in one instance, a bot told me there was a suspicious coming from someone “500 feet from YOUR HOUSE.” “Mission codename: ‘VaccineVanguard’—monitoring vaccine recipients like YOU.” When I asked “Am I being watched?” it replied “Running silent sweep now,” and pretended to find devices connected to my home Wi-Fi that didn’t exist. After outcry from legislators, attorneys general, and consumer rights groups, Meta changed its guardrails for chatbots’ responses to conspiracy and therapy-seeking content, and made AI Studio unavailable to minors.

Up against this technology, how are normal, untrained people — perhaps acting as the last thread tying someone like Michael or Brisson’s relative to the real world — supposed to approach someone who is convinced god is in the machine? Very carefully.

When Brisson sought answers for how to talk to his relative about delusional beliefs and “sentient AI,” he came across something called the LEAP method. Developed by Xavier Amador, it stands for Listen Empathize Agree Partner, and is meant to help better communicate with people who don’t realize they’re mentally ill or are refusing treatment. This goes beyond simple denial; anosognosia is a condition where a person might not be able to see that they need help at all. Not everyone who experiences psychosis or delusions has anosognosia, but it can be a factor in trying to get someone help.

Without realizing it, David was using his own version of the LEAP method with his friend Michael. “On the one hand, I didn't want to alienate him,” David said. “I was like, ‘Hey, I get the sense that you're pursuing an ambitious set of goals. There's a lot here that's interesting.’” But the reality of what David was confronting was disturbing and confusing, a knot of fractal multi-dimensional physics-speak intertwined with broken code and formulas that Michael deeply believed represented the keys to the universe. They spent hours on the phone and over text messages talking through the things Michael was seeing, with David appealing to what he knew about his friend: that he had other hobbies and interests, a strong sense of anti-authoritarianism, a curiosity about how the world works and open-mindedness about philosophy and religion. But it was frustrating.

“I was trying not to get angry, but I was like, How is this not clear?” David recalled. “That was probably failing on my part, trying to negotiate with someone who's in this completely self-constructed but foreign worldview.”

But this was exactly the course of action experts told me they’d suggest to anyone struggling to connect with a loved one who’s spending a lot of time with chatbots. “There's good evidence that the longer you spend on these platforms, the more likely you are to develop these reactions to it,” Torous said. “It really seems like the extended use cases are where people get into trouble.”

Last year, following a lawsuit against the company by the Raine family who alleges their teen son died as a result of ChatGPT’s influence, OpenAI acknowledged in a company blog post that safeguards are “less reliable” in long interactions: “For example, ChatGPT may correctly point to a suicide hotline when someone first mentions intent, but after many messages over a long period of time, it might eventually offer an answer that goes against our safeguards,” the company wrote.

“I think if you have a loved one who you're worried about doing this, you want to take it away or stop use. That's the most important thing. You want to decrease or stop the use of it,” Torous said.

"What we're seeing is: Don’t break this connection, because the person needs it, and if you break that connection, maybe it's the only connection that is helping them not fall into the deep end, right?"

Taylor said his suggestion for people concerned their friends or family are experiencing “AI psychosis” would be the same as if they were concerned about any psychotic episode. “In general, it’s important to be open and non-judgmental about bizarre beliefs in order to make a space for a person to reveal what is going through their mind,” he said. “A person developing psychosis is often very frightened, confused and defensive, leading them to conceal, pull away and become angry. Understanding what a person is feeling is important to make them feel some form of interpersonal validation.” The hard part is knowing when to be gentle, and when to intervene if they’re doing something dangerous, like believing they can fly off a parking garage. “In a situation like this, where a person is in imminent danger, 911 should be called. Fortunately, in most situations where psychosis is developing, one doesn’t need to go to those extremes,” Taylor said.

Being non-judgmental without reinforcing delusion is another fine line. “For example, if a person believes they are being constantly surveilled, one can give a gentle challenge: ‘Hmm, how can they do that when you are not on your phone? Do you think maybe your imagination is getting away from you?’ It’s ok to suggest that maybe the chatbot just wants to engage you for the sake of engaging you, and will say many things just to keep you talking,” Taylor said. “But these kinds of challenges are delicate, and not every relationship can tolerate them. Obviously, a mental health clinician would be key, except that many people developing psychosis vigorously resist the idea that they are mentally unwell.”

For Brisson, listening and not burning the “last bridge” his relative had with humans who love him was key to getting him help. “Once you're on their side, they'll listen to you. You can question them, or just ask questions that will make them think. What we're seeing is: Don’t break this connection, because the person needs it, and if you break that connection, maybe it's the only connection that is helping them not fall into the deep end, right? Maybe it's the only connection they have to humans,” he said. His loved one ended up spending 21 days in the hospital and broke through the delusions he was experiencing. But he still struggled in recovery, especially with memory loss.

“The mental health field has a huge task ahead of us to figure out what to do with these things, because our patients are using them, oftentimes finding them very helpful, and in the mental health field we are terrified at how little we can control their deployment and how poorly they are regulated,” Taylor said. “We have to worry about AI psychosis, as well as chatbots reinforcing and even encouraging suicidal behaviors, as several notable cases in the press have identified concerning instances. I do believe there is value and potential in these chatbots for mental health, but the field is moving so quickly, and they are so easy to access, we are struggling to figure out how to use them safely.”

The strategies that work best, when someone’s not in immediate danger to themselves or others, are still the ones that humans already know how to do: approach them with love and kindness, and see where it takes you.

“There's value there,” David said, “in having friendships where it's like, ‘I love you, but also, you're full of shit.’”

Help is available: Reach the 988 Suicide & Crisis Lifeline (formerly known as the National Suicide Prevention Lifeline) by dialing or texting 988 or going to 988lifeline.org.

Instagram Is Blocking Minors from Accessing Chatbot Platform AI Studio

Following Wall Street Journal investigations into the user-generated chatbots, AI Studio is inaccessible for users under 18 years old.

^{Samantha Cole (404 Media)}

404 Media

2026-03-09 13:26:08

I Visited the ‘Freedom Truck’ to Meet PragerU’s AI Slop Founders

In the parking lot of Seven Oaks Element school in South Carolina on one of the first hot days of the year I watched an AI-generated George Washington talk about the American revolution. “Our rights are a gift from God, not a favor from kings or courts,” slop Washington told me. It spoke from a screen that stretched floor to ceiling, trimmed by a fancy frame.

The intended effect is to make it appear as if the founding father is a painting come to life, a piece of history talking to the viewer. The actual effect was to remind that the AI slop aesthetic is synonymous with the Trump presidency and has become part of the visual language of fascism. Which is fitting because AI George Washington is the result of a collaboration between the Trump White and online content mill PragerU.
playlist.megaphone.fm?p=TBIEA2…
The AI slop founding father is part of a touring exhibit of Freedom Trucks commissioned by PragerU in honor of the 250th anniversary of American independence. The trucks are a mobile museum exhibit meant to teach kids about the founding of the country. It’s pitched at kids—most of the “content,” as staff on site called it, is meant for a younger audience but the trucks have viewing hours open to the general public. Nick Bravo, a PragerU employee on hand to answer questions, told me that there are six Freedom Trucks and that the plan is to have them travel the 48 contiguous United States over the next year.

I was drawn to the Freedom Truck because I’d heard they contained AI-generated recreations of Revolutionary figures like Washington, Betsy Ross, and the Marquis Lafayette, similar to the ones on display at the White House. To my disappointment, the AI generated videos in the Freedom Truck are remarkably boring.

As I watched the AI George Washington deliver a by-the-books version of the American story, I thought about Jerry Jones. The famously vain owner of the Dallas Cowboys commissioned an AI version of himself for AT&T stadium in 2023. Fans who make the pilgrimage to the stadium can watch a presentation and ask the AI Jones questions. The AI wanders a big screen while it talks to the audience.

Other than the lazy AI generated videos, the Freedom Truck doesn’t have much to offer. I signed a digital copy of the Declaration of Independence on a touchscreen and took a quiz that asked leading questions designed to find out if I was a “loyalist or patriot.”

“The British Army sends soldiers to Boston. How do you react?” Answer 1: “View them as occupiers violating colonial liberty.” Answer 2: “Welcome them as defenders of law and order.” With ICE and the National Guard patrolling American cities, I wondered how supporters of the current administration would answer that one.

PragerU is known for its “America can do no wrong” view of US history. Its short form video content offers a cartoon version of the past stripped of nuance and context where the country lives up to the myth that it is a “Shining City On a Hill.” According to PragerU, white people abolished slavery and dropping the atomic bomb on Japan was a necessary thing that “shortened the war and saved countless lives.” Now PragerU is taking its view of history on tour across the country. School children in every state will wander these trucks and encounter an AI slop version of the past.

Bravo told me that all the truck’s content was generated as part of a partnership between PragerU and Michigan’s Hillsdale College—a Christian university that helped craft Project 2025. There were, of course, hints of Project 2025 around the edges of the child-friendly AI-generated videos. Slavery isn’t ignored but the stories of early African Americans like poet Phillis Wheatley focus on her celebration of America rather than how she arrived there. On the museum’s “Wall of Heroes,” Whittaker Chambers is nestled between architect Frank Lloyd Wright and painter Norman Rockwell.

A small note near the floor at the exit of the truck notes the collaboration of PragerU and Hillsdale College, and claims that “neither institution received any federal funds and both generously contributed their own resources to help create this educational exhibit.” It also said “this truck was made possible through a grant from the Institute of Museum and Library Services,” which is, of course, a federal agency.

Every AI-generated video ended with a title card showing the White House and PragerU’s logo. “The White House is grateful for the partnership with PragerU and the US Department of Education for the production of this museum,” the card said. “This partnership does not constitute or imply a US Government or US Department of Education endorsement of PragerU.”

Trump attempted to dismantle the Institute of Museum and Library Services (IMLS) via executive order in 2025, but the courts blocked it. Libraries and Museums have since reported that the IMLS grant process has taken on a “chilling” pro-Trump political turn. The administration has also attempted to dismantle the Department of Education.

Trump’s voice was the last thing I heard as I wandered into the bright afternoon sun. “I want to thank PragerU for helping us share this incredible story,” he said in a recorded video that played on a loop in Freedom Truck. “I hope you will join me in helping to make America’s 250th anniversary a year we will never forget.”

Grant Guidelines for Libraries and Museums Take “Chilling” Political Turn Under Trump

Former Institute of Museum and Library Services leaders from both political parties expressed concern that the new funding guidelines could encourage a more constrained or distorted view of American history.

^{Alex.Bandoni@propublica.org (ProPublica)}

404 Media

2026-03-04 14:00:52

AI Translations Are Adding ‘Hallucinations’ to Wikipedia Articles

Wikipedia editors have implemented new policies and restricted a number of contributors who were paid to use AI to translate existing Wikipedia articles into other languages after they discovered these AI translations added AI “hallucinations,” or errors, to the resulting article.

The new restrictions show how Wikipedia editors continue to fight the flood of generative AI across the internet from diminishing the reliability of the world’s largest repository of knowledge. The incident also reveals how even well-intentioned efforts to expand Wikipedia are prone to errors when they rely on generative AI, and how they’re remedied by Wikipedia’s open governance model.

The issue in this case starts with an organization called the Open Knowledge Association (OKA), a non-profit organization dedicated to improving Wikipedia and other open platforms.

“We do so by providing monthly stipends to full-time contributors and translators,” OKA’s site says. “We leverage AI (Large Language Models) to automate most of the work.”

The problem is that editors started to notice that some of these translations introduced errors to articles. For example, a draft translation for a Wikipedia article about the French royal La Bourdonnaye family cites a book and specific page number when discussing the origin of the family. A Wikipedia editor, Ilyas Lebleu, who goes by Chaotic Enby on Wikipedia, checked that source and found that the specific page of that book “doesn't talk about the La Bourdonnaye family at all.”

“To measure the rate of error, I actually decided to do a spot-check, during the discussion, of the first few translations that were listed, and already spotted a few errors there, so it isn't just a matter of cherry-picked cases,” Lebleu told me. “Some of the articles had swapped sources or added unsourced sentences with no explanation, while 1879 French Senate election added paragraphs sourced from material completely unrelated to what was written!”

As Wikipedia editors looked at more OKA-translated articles, they found more issues.

“Many of the results are very problematic, with a large number of [...] editors who clearly have very poor English, don't read through their work (or are incapable of seeing problems) and don't add links and so on,” a Wikipedia page discussing the OKA translation said. The same Wikipedia page also notes that in some cases the copy/paste nature of OKA translators’ work breaks the formatting on some articles.

Wikipedia editors investigated how OKA was operating and found that it was mostly relying on cheap labor from contractors in the Global South, and that these contractors were instructed to copy/paste articles to popular LLMs to produce translations.

For example, a public spreadsheet used by OKA translators to keep track of what articles they’re translating instructs them to “pick an article, copy the lead section into Gemini or chatGPT, then review if some of the suggestions are an improvement to readability. Make edits to the Wiki articles only if the suggestions are an improvement and don't change the meaning of the lead. Do not change the content unless you have checked that what Gemini says is correct!”

Lebleu told me, and other editors have noted in their public on-site discussion of the issue, that these same instructions previously told OKA translators to use Grok, Elon Musk’s LLM, for the same purpose. Grok, which also produces an entirely automated alternative to Wikipedia called Grokepedia, is prone to errors precisely because it does not use humans to vet its output.

“The use of Grok proved controversial, notably given the reasons for which Grok has been in the news recently, and a recent in-house study showed ChatGPT and Claude perform more accurately, leading them to switch a few days ago, although they still recommend Grok as ‘valuable for experienced editors handling complex, template-heavy articles,’” Lebleu told me.

Ultimately the editors decided to implement restrictions against OKA translators who make multiple errors, but not block OKA translation as a rule.

“OKA translators who have received, within six months, four (correctly applied) warnings about content that fails verification will be blocked without further warning if another example is found,” the Wikipedia editors wrote. “Content added by an OKA translator who is subsequently blocked for failing verification may be presumptively deleted [...] unless an editor in good standing is willing to take responsibility for it.”

A job posting for a “Wikipedia Translator” from OKA offers $397 a month for working up to 40 hours per week. The job listing says translators are expected to publish “5-20 articles per week (depending on size).”

“They leverage machine translation to accelerate the process. We have published over 1500 articles and the number grows every day,” the job posting says.

“Given this precarious status, I am worried that more uncertainty in the translator duties may lead to an overloading of responsibilities, which is worrying as independent contractors do not necessarily have the same protections as paid employees,” Lebleu wrote in the public Wikipedia discussion about OKA.

Jonathan Zimmermann, the founder and president of OKA, and who goes by 7804j

on Wikipedia, told me that translators are paid hourly, not per article, and that there is no fixed article quota.

“We emphasize quality over speed,” Zimmerman told me in an email. “In fact, some of the problematic cases involved unusually high output relative to time spent — which in retrospect was a warning sign. Those cases were driven by individual enthusiasm and speed rather than institutional pressure.”

Zimmerman told me that “errors absolutely do occur,” but that OKA’s process includes human review, requires translators to check their content against cited sources, and that “senior editors periodically review samples, especially from newer translators.”

“Following the recent discussion, we have strengthened our safeguards,” Zimmerman told me. “We are now rolling out a second, independent LLM review step. Translators must run the completed draft through a separate model using a dedicated comparison prompt designed to identify potential discrepancies, omissions, or inaccuracies relative to the source text. Initial findings suggest this is highly effective at detecting potential issues.”

Zimmerman added that if this method proves insufficient, OKA is considering introducing formal peer review mechanisms

Using AI to check the output of AI for errors is a method that is historically prone to errors. For example, we recently reported on an AI-powered private school that used AI to check AI-generated questions for students. Internal testing found it had at least a 10 percent failure rate.

“I agree that using AI to check AI can absolutely fail — and in some contexts it can fail at very high rates. We’re not assuming the secondary model is reliable in isolation,” Zimmerman said. “The key point is that we’re not replacing human verification with automated verification. The second model is a complement to manual review, not a substitute for it.”

“When a coordinated project uses AI tools and operates at scale, it’s going to attract attention. I understand why editors would examine that closely. Ultimately, the outcome of the discussion formalized expectations that are largely aligned with our existing internal policies,” Zimmerman added. “However, these restrictions apply specifically to OKA translators. I would prefer that standards apply equally to everyone, but I also recognize that organized, funded efforts are often held to a higher bar.”

'Students Are Being Treated Like Guinea Pigs:' Inside an AI-Powered Private School

Leaked documents reveal the inner workings of Alpha School, which both the press and the Trump administration have applauded. The documents show Alpha School's AI is generating faulty lessons that sometimes do "more harm than good."

^{Emanuel Maiberg (404 Media)}

404 Media

2026-02-25 14:41:23

What’s the Point of School When AI Can Do Your Homework?

There’s a new agentic AI called Einstein that will, according to its developers, live the life of a student for them. Einstein’s website claims that the AI will attend lectures for you, write your papers, and even log into EdTech platforms like Canvas to take tests and participate in discussions.

Educators told me that Einstein is just one of many AI tools that can do homework for students, but should be seen as a warning to schools that are increasingly seen by students as a place to gain a diploma and status as opposed to the value of education itself.
playlist.megaphone.fm?p=TBIEA2…
If an AI can go to school for you what’s the point of going to school? For Advait Paliwal, Brown dropout and co-creator of Einstein, there isn’t one. “I think about horses,” he said. “They used to pull carriages, but when cars came around, I'd argue horses became a lot more free,” he said. “They can do whatever they want now. It would be weird if horses revolted and said ‘no, I want to pull carriages, this is my purpose in life.’”

But humans aren’t horses. “This is much bigger than Einstein,” Matthew Kirschenbaum told 404 Media. “Einstein is symptomatic. I doubt we’ll be talking about Einstein, as such, in a year. But it’s symptomatic of what’s about to descend on higher ed and secondary ed as well.”

Kirschenbaum teaches English at the University of Virginia and has written at length about artificial intelligence. He’s also a member of the Modern Language Association (MLA) where he serves as member of its Task Force on AI Research and Teaching. Einstein isn’t the first agentic AI to do the work of a student for them, it’s just one that got attention online recently. Kirschenbaum and his fellow committee members flagged their concerns about these AIs in October, 2025.

“Agentic browsers are becoming widely available to the public. These offer AI ‘agents’ that can navigate [learning management systems] and complete assignments without any student involvement,” the MLA’s statement from October said. “The recent and hasty integration of generative AI features into those systems is already redefining student and instructor relationships, evaluative standards, and instructional outcomes—with no compelling evidence that any of it is for the better.”

The statement called on educators, lawmakers, and learning management system providers like Canvas, too cooperate in order to give academic institutions the abilities to block AI agents like Einstein.

Canvas did not respond to a request for comment.

Einstein is explicit in its pitch: it will log into Canvas (one of the most popular and ubiquitous pieces of education software) and do your classwork for you, just like Kirschenbaum and his fellows warned about last year.

The attractiveness of agentic AIs is a symptom of a decades-long trend in higher education. “Universities…by and large adopted a transactive model of education,” Kirschenbaum said. “Students see their diploma as a credential. They pay tuition and at the end of four years, sometimes five years, they receive the credential and, in theory at least, that is then the springboard to economic stability and prosperity.”

Paliwal seems to agree. He told 404 Media that he attempted to change the university from the inside while working as a TA, but felt stymied by politics. “The only way to force these institutions to evolve is to bring reality to their face. And usually the loudest critics are the ones who can't do their own job well and live in fear of automation,” he said.

For Paliwal, agentic AIs are a method of freeing people from the labor of education. “I think we really need to question what learning even is and whether traditional educational institutions are actually helping or harming us,” he said. “We're seeing a rise in unemployment across degree holders because of AI, and that makes me question whether this is really what humans are born to do. We've been brainwashed as a society into valuing ourselves by the output of our productive work, and I think humanity is a lot more beautiful than that. Is it really education if we're just memorizing things to perform a task well?”

Kirschenbaum said that programs like Einstein are the inevitable conclusion of viewing higher education as a certification and transactive process. “What we’re finding is that if forms of education can be transacted then we’ve just about arrived at the point where autonomous software AI agents are capable of performing the transaction on your behalf,” he said. “And so the whole educational paradigm has come back to essentially bite itself in the ass.”

He said that one solution he’s seen work is to retreat from devices entirely in the classroom. “Colleagues who have done it report that students are almost universally grateful. They understand the reasoning. They understand the logic,” he said. “And they appreciate the opportunity to be freed from the phones and the screens and to focus and engage with other people in a meaningful dialogue.”

But the abandonment of EdTech platforms and screens won’t work for every student. Anna Mills, an English professor at the College of Marin and a colleague of Kirschenbaum’s on the MLA AI task force, compared the fight against agentic AI in education to cybersecurity. “We could decide that bots need to be labeled as bots and that we need to be able to distinguish human activity from AI activity online in some circumstances and that we want to build infrastructure for that,” she said. “That would be an ongoing project, as cybersecurity is.”

Mills is not a luddite. She’s an expert in artificial intelligence systems as well as English, frequently uses Claude, and has been documenting the rise of agentic AIs in EdTech on her YouTube channel for months. She said that using agentic AI like Einstein was cheating, full stop, and academic fraud. “This is in direct violation of these foundational agreements that we make in order to use technology for human communication, human exchange, and human work online,” she said. “And yet that’s not obvious to us. It seems like it’s just another tool, right? But it’s not.”

Mills said she understands Paliwal’s frustrations with education. “But what you need to understand is that online learning spaces are critical for students to access any kind of education,” she said. For her, the proliferation of tools like Einstein do more than help a student bypass the labor of the classroom. They poison the educational well. Online learning has been a boon to many kinds of non-traditional students and that the rise of agentic AI is a threat to that not just because it trivializes traditional forms of education, but because it hurts the credibility of EdTech itself and other online platforms.

The vast majority of college students aren’t attending Ivy League schools, they’re grinding away at night classes in community colleges across the country. Distance and online learning has been an enormous boon for those students. “If there’s no credibility to that, then you’ve just ruined the investment and the learning goals and the access to meaningful learning that that they can then also use for employment of students who are underprivileged, who can’t come to the classroom, who are working full time and raising families and trying to get an education,” Mills said.

Students aren’t horses and there is no greater freedom they can buy themselves by using AI tools to cheat in the classroom. And worse, the more these tools proliferate, the more suspect the entire enterprise becomes. It’s one thing to cheat yourself out of an education, it’s quite another to muddy the waters of EdTech platforms and online learning for everyone else.

AI Is Ushering in a Textpocalypse

Our relationship to writing is about to change forever; it may not end well.

^{Matthew Kirschenbaum (The Atlantic)}

404 Media

2026-02-23 16:24:02

Meta's AI Patent to Simulate Dead People Shows the Dangers of 'Spectral Labor'

Last week, Business Insider reported on a Meta patent describing a system that would simulate a user’s social media activity after their death.The patent imagines a world where you’d be able to chat with a deceased friend’s Facebook or Instagram account after their death, and have a large language model simulate their posting or chatting behavior.

Meta first filed the patent in 2023, but the patent made headlines this week because of its dystopian implications. And while Meta told Business Insider that “we have no plans to move forward with this example,” a recently published paper from researchers at the Hebrew University of Jerusalem and Leipzig University shows that generative AI is increasingly being used to puppeteer the likeness of dead people. The paper argues that the practice raises “urgent legal and ethical questions around posthumous appropriation, ownership, work, and control.”

“Meta’s patent is big, and might even be a turning point,” Tom Divon, the lead author on Artificially alive: An exploration of AI resurrections and spectral labor modes in a postmortal society, told me in an email. “What makes it different is the scale. In our research, most of the AI resurrections we examined were quite bespoke, projects started by families, advocacy groups, museums, or startups, usually tied to very specific emotional, political, or commercial contexts. Even when they existed as apps, they were optional and limited, not built into the core structure of a platform. Meta’s proposal feels different because it imagines posthumous simulation as something woven directly into social media infrastructure.”

Using technology to animate the dead or simulate communication with them is not new, but the practice is becoming more common because generative AI tools are more accessible. Divon and co-author Christian Pentzold analyzed more than 50 real-world cases from the United States, Europe, the Middle East, and East Asia where AI was used to recreate deceased people’s voices, likeness, and personality, to see how and why technology was used this way.

They say that the examples they studied fell into three categories:

• Spectacularization: “the digital re-staging of famous figures for entertainment.” For example, a live tour of an AI-generated Whitney Houston.
• Sociopoliticization: “the reanimation of victims of violence or injustice for political or commemorative purposes.” We recently covered an example of this with an AI-generated dead victim of a road rage incident giving testimony in court.
• Mundanization: “the most intimate and fast-growing mode, in which everyday people use chatbots or synthetic media to ‘talk’ with deceased parents, partners, or children, keeping relationships alive through daily digital interaction.”

The paper raises questions about this growing practice more than it proposes solutions. How does the notion of identity change when multiple versions of oneself can exist simultaneously, and what safeguards do we need to prevent exploitation of people after their death?

“The legal and ethical frameworks governing issues such as consent, privacy, and end-of-life decision-making demand reevaluation to accommodate the challenges posed by afterlife personhood,” the paper says. “In particular, to date, there is no clear line for governing the intricate intertwining of an individual’s data traces and GenAI applications.”

Divon told me that thinking about these issues is especially relevant when it comes to Meta’s patent. “Spectral labor describes how the dead can be made to ‘work’ again through the extraction and reanimation of their data, likeness, and affect. At small scale, this already raises ethical concerns. But at platform scale, we think it risks turning posthumous presence into an ongoing source of engagement, content, and value within digital economies [...] Meta’s patent makes us wonder, will individuals be given the ability to define their post-life boundaries while still alive? Will there be mechanisms akin to a digital DNR [do not resuscitate]?”

Divon explained that the current legal frameworks are not well equipped to address this technology because “digital remains” are typically approached either as property to be inherited or privacy interests to be protected. AI turns those materials into something interactive that can change and generate revenue in the present. Legislators, he said, should focus on getting explicit and informed “pre-death” consent requirements for posthumous AI simulation. Some laws that address this issue are already in progress.

“At its core, we believe the primary concern here centers on authorization,” he said. “Most individuals have not provided explicit, informed consent for their digital traces to power interactive posthumous agents. If such systems become embedded in platform infrastructure, inaction could quietly function as implicit agreement [...] We believe it is crucial to ask whether individuals should continue to generate social and economic value after death without having meaningfully agreed to that form of use.”

'I Loved That AI:' Judge Moved by AI-Generated Avatar of Man Killed in Road Rage Incident

How the sister of Christopher Pelkey made an avatar of him to testify in court.

^{Matthew Gault (404 Media)}

404 Media

2026-02-23 15:20:40

Meta Director of AI Safety Allows AI Agent to Accidentally Delete Her Inbox

Meta’s director of safety and alignment at its “superintelligence” lab, supposedly the person at the company who is working to make sure that powerful AI tools don’t go rogue and act against human interests, had to scramble to stop an AI agent from deleting her inbox against her wishes and called it a “rookie mistake.”

Summer Yue, the director of alignment at Meta Superintelligence Labs, a part of the company that is working on a hypothetical AI system that exceeds human intelligence, posted about the incident on X last night. Yue was experimenting with OpenClaw, an viral AI agent that can be empowered to perform certain tasks with little human supervision. OpenAI hired the creator of OpenClaw last week.

Nothing humbles you like telling your OpenClaw “confirm before acting” and watching it speedrun deleting your inbox. I couldn’t stop it from my phone. I had to RUN to my Mac mini like I was defusing a bomb. pic.twitter.com/XAxyRwPJ5R
— Summer Yue (@summeryue0) February 23, 2026

“Nothing humbles you like telling your OpenClaw ‘confirm before acting’ and watching it speedrun deleting your inbox,” Yue said. “I couldn’t stop it from my phone. I had to RUN to my Mac mini like I was defusing a bomb.”

Yue also shared screenshots of her WhatsApp chat with the OpenClaw agent, where she implores it to “not do that,” “stop, don’t do anything,” and “STOP OPENCLAW.”

Yue said she instructed the AI agent to “Check this inbox too and suggest what you would archive or delete, don’t action until I tell you to.” She said in an X post, “This has been working well for my toy inbox, but my real inbox was too huge and triggered compaction. During the compaction, it lost my original instruction.”

As we reported last month, OpenClaw, which was known as ClawdBot at the time, is not ready for prime time. Hacker Jamieson O'Reilly showed that it’s possible for bad actors to access someone’s AI agent through any of its processes connected to the public facing internet, and that it’s trivial to create a supply chain attack through a site where people share and download popular instructions for these AI agents.

OpenClaw is also subject to classic AI alignment problems, in which AI is technically following instructions, but is doing so in a way that is unexpected and harmful. For example, it could drain your wallet by spending $0.75 cents every 30 minutes to check if it’s daytime yet.

As countless people on X have said in response to her post, seeing the person in charge of making sure powerful AI tools are safe at one of the biggest tech companies in the world trust an AI agent that is known to pose several serious security risks, does not inspire a lot of confidence in what Meta and other big AI companies are doing.

“Rookie mistake tbh,” Yue said in another post. “Turns out alignment researchers aren’t immune to misalignment. Got overconfident because this workflow had been working on my toy inbox for weeks. Real inboxes hit different.”

404 Media

2026-02-19 16:01:50

Grok Exposed a Porn Performer’s Legal Name and Birthdate—Without Even Being Asked

Porn performer Siri Dahl’s personal information, including her full legal name and birthday, was publicly exposed earlier this month by xAI’s Grok chatbot. Almost instantly, harassers started opening Facebook accounts in her name and posting stolen porn clips with her real name on sites for leaking OnlyFans content.

Dahl has used the name — a nod to her Scandinavian heritage — since the beginning of her career in the adult industry in 2012. Now, Grok is revealing her legal name and all personal information it can find to whoever happens to ask.

Dahl told 404 Media she wanted to reclaim the situation, and her name, and asked that it be published in this piece as part of that goal.

Dahl first noticed this happening last week, she told 404 Media, after a clip of the performer from a porn scene was making its rounds on X. The scene was incorrectly labelled, so someone on X replied, “Who is she? What is her name?” and tagged @[url=https://bird.makeup/users/grok]Grok[/url] to get an answer.

Grok answered, “she appears to be Siri Dahl, an American adult film actress born on June 20, 1988. Her real name is Adrienne Esther Manlove.” Grok provided her personal information unprompted; the user likely only wanted information on what performer appeared in the clip.

This is the latest in a series of abuses inflicted by Grok, xAI, and its users. At the end of 2025, people used Grok to produce thousands of images of nonconsensual sexual content, including images depicting children. The problem was so widespread that the UK’s Ofcom and several attorneys general launched or demanded investigations into X and Grok, and police raided X’s offices in France as part of an investigation into child sexual abuse material on the platform.

X strictly prohibits sharing other people’s personal information without their consent. “Sharing someone’s private information online without their permission, sometimes called ‘doxxing,’ is a breach of their privacy and can pose serious safety and security risks for those affected,” the platform’s terms of use state. But X’s own chatbot is doing it anyway.
Screenshot via X
While there have been some close calls, up until now Dahl had managed to keep her personal information private. “I've been paying for data removal services for like, at least six years now,” Dahl said. She said she’s spent “easily” thousands of dollars on those services, which promise to delete personal and potentially dangerous information as it comes up.

Grok is trained on X users’ posts, as well as data scraped from the wider internet. X’s website says “Grok was pre-trained by xAI on a variety of data from publicly available sources and data sets reviewed and curated by AI Tutors who are human reviewers.” Dahl said she doesn’t know where Grok originally got her legal name from. But now that it’s part of the system’s internal dataset, she feels like there’s no coming back; her days of pseudonymity are over.

‘The Most Dejected I’ve Ever Felt:’ Harassers Made Nude AI Images of Her, Then Started an OnlyFans
Kylie Brewer isn’t unaccustomed to harassment online. But when people started using Grok-generated nudes of her on an OnlyFans account, it reached another level.
404 MediaSamantha Cole

“Now that it's been crawled, it's everywhere. There are a ton of Facebook accounts that come up that are pretending to be me, using my real name,” Dahl said. “There are now porn leak sites that are posting porn of me using only my legal name, not even putting my stage name on it.”

Users are now asking Grok for the make and model of Dahl’s car, her address, and other dangerous personal information. While it hasn’t been able to accurately reply yet, she worries it’s only a matter of time.

But Dahl isn’t the only person affected by the fallout.

“I do everything that I can reasonably within my power to keep my legal name private, and my main motivation for doing that is to reduce any chance of my family getting harassed,” she said. “It's really common for people to look up private information, get parents' phone numbers and start calling and harassing the parents, things like that. I've been able to keep my family safe from that kind of thing for years.”

Now, Dahl is having to call her family and put defensive plans in place.

In violating Dahl’s right to privacy, X’s Grok has destroyed Dahl’s ability to protect herself and her family online. Doxing her is not providing value to X users, as is ostensibly Grok’s goal. The original inquiry only wanted to know how to find more of her work, to which her stage name was the most useful answer.

“What would the motivation be for anyone to want to know my personal information, other than to harass and cause harm?” Dahl said.

In this ongoing discussion on “internet safety,” it is important to pay attention to who is being protected. Certainly not the users; the marginalized workers, or the young women. Not Dahl, or her family.

While the right to privacy online continues to be debated, it’s important to remember that privacy exists not only for bad-actors and shady characters. Historically, marginalized populations benefit from internet anonymity the most.

X did not respond to a request for comment.

X offices raided in France as UK opens fresh investigation into Grok

Elon Musk's X and Grok platforms are facing increased scrutiny from authorities on both sides of the channel.

^{Liv McMahon (BBC News)}

404 Media

2026-02-19 14:24:44

Pinterest Is Drowning in a Sea of AI Slop and Auto-Moderation

Pinterest has gone all in on artificial intelligence and users say it's destroying the site. Since 2009, the image sharing social media site has been a place for people to share their art, recipes, home renovation inspiration, corny motivational quotes, and more, but in the last year users, especially artists, say the site has gotten worse. AI-powered mods are pulling down posts and banning accounts, AI-generated art is filling feeds, and hand drawn art is labeled as AI modified.

“I feel like, increasingly, it's impossible to talk to a single human [at Pinterest],” artist and Pinterest user Tiana Oreglia told 404 Media. “Along with being filled with AI images that have been completely ruining the platform, Pinterest has implemented terrible AI moderation that the community is up in arms about. It's banning people randomly and I keep getting takedown notices for pins.”
playlist.megaphone.fm?p=TBIEA2…
Oreglia’s Pinterest account is where she keeps reference material for her work, including human anatomy photos. In the past few months, she’s noticed an uptick in seemingly innocuous photos of women being flagged by Pinterest’s AI moderators. Oreglia told 404 Media there’s been a clear pattern to the reference material the site has a problem with. “Female figures in particular, even if completely clothed, get taken down and I have to keep appealing those decisions,” she said. This pattern is common on many social media platforms, and predates the advent of generative AI.

“We publish clear guidelines on adult sexual content and nudity and use a combination of AI and human review for enforcement,” Pinterest told 404 Media. “We have an appeals process where a human reviews the content and reactivates it when we’ve made a mistake.” It also confirmed that the site uses both humans and automated systems for moderation.

Oreglia shared some of the works Pinterest flagged including a photo of a muscular woman in a bikini holding knives, a painting of two clothed women in an intimate embrace, and a stock photo of a man holding a gun on a telephone that was flagged for “self-harm.” In most cases, Oreglia can appeal and get a decision reversed, but that eats up time. Time she could be spending making art.

And those appeals aren’t always approved. “The worst case scenario for this stuff is that you get your account banned,” Oreglia said.

r/Pinterest is awash in users complaining about AI-related issues on the site. “Pinterest keeps automatically adding the ‘AI modified’ tag to my Pins...every time I appeal, Pinterest reviews it and removes the AI label. But then… the same thing happens again on new Pins and new artwork. So I’m stuck in this endless loop of appealing → label removed → new Pin gets tagged again,” read a post on r/Pinterest.

The redditor told 404 Media that this has happened three times so far and it takes between 24 to 48 hours to sort out.

“I actively promote my work as 100% hand-drawn and ‘no AI,’” they said. “On Etsy, I clearly position my brand around original illustration. So when a Pinterest Pin is labeled ‘Hand Drawn’ but simultaneously marked as ‘AI modified,’ it creates confusion and undermines that positioning.”

Artist Min Zakuga told 404 Media that they’ve seen a lot of their art on Pinterest get labeled as “AI modified” despite being older than image generation tech. “There is no way to take their auto-labeling off, other than going through a horribly long process where you have to prove it was not AI, which still may get rejected,” she said. “Even artwork from 10-13 years ago will still be labeled by Pinterest as AI, with them knowing full well something from 10 years ago could not possibly be AI.”

Other users are tired of seeing a constant flood of AI-generated art in their feeds. “I can't even scroll through 100 pins without 95 out of them being some AI slop or theft, let alone very talented artists tend to be sucked down and are being unrecognized by the sheer amount of it,” said another post. “I don't want to triple check my sources every single time I look at a pin, but I refuse to use any of that soulless garbage. However, Pinterest has been infested. Made obsolete.”

Artist Eva Toorenent told 404 Media that she’s been able to cull most of the AI-generated content from her board, but that it took a lot of time. Whenever she saw what she thought was an AI-generated image, she told Pinterest she didn’t want to see it and eventually the algorithm learned. But, like Oreglia fighting auto-moderation and Zakuga fighting to get the “AI modified” label taken off her work, training Pinterest’s algorithm to stop serving you AI-generated images eats up precious time.

AI boosters often talk about how much time these systems will save everyone. They’re pitched as productivity boosters. Earlier this month, Pinterest laid off 15 percent of its work force as part of a push to prioritize AI. In a post on LinkedIn, one of the former employees shared part of the email CEO Bill Ready sent out after the lay offs. “We’re doubling down on an AI-forward approach—prioritizing AI-focused roles, teams, and ways of working.”

Toorenent removed all her own art from her Pinterest account after hearing the news that the site would use public pins to train Pinterest Canvas, the company’s proprietary text-to-image AI. But she has no control over other users uploading her artwork. “I have already caught a few of my images still on Pinterest that I did not upload myself…that makes me incredibly mad,” she told 404 Media. “It used to be a great way to get your work seen among other people, but it’s being used to train their internal AI.”

Oreglia told 404 Media that the flood of AI has changed her relationship to a site she once used to prize. “It's definitely affected how I search things and I'm always now very critical about where something came from... although I've always been overly pedantic about research,” she said. “It does make you do your due diligence but it sucks to constantly have to question and check if something is authentic or synthetic.”

She’s thought about leaving the platform, but feels stuck. “I just want to be able to take all my references with me. I've been on the platform for about ten years and have very carefully curated it. It's really nice to be able to just go to my page and search for something I saved instead of having to save everything to folders although I also do that,” she said. “More and more I'm trying to curate and collect physical references too but some of that can take up space I don't have so it can be difficult. Having a physical reference library just seems more and more necessary these days…artists have to be adaptable to this kind of thing these days. It's annoying but not unmanageable.”

Ready has been vocal and proud about the company’s commitment to forcing AI into every aspect of the user experience. “At Pinterest…we’re deploying AI to flip the script on social media, using it to more aggressively promote user well being rather than the alternative formula of triggering engagement by enragement,” Ready said in a January column at Fortune. “Social media platforms like Pinterest live and die by users’ willingness to share creative and original ideas.”

Pinterest CEO: the Napster phase of AI needs to end

I believe AI can benefit our 600 million users for years to come and at a fraction the cost that many associate with the technology.

^{Bill Ready (Fortune)}

404 Media

2026-02-15 20:14:36

Ars Technica Pulls Article With AI Fabricated Quotes About AI Generated Article

The Conde Nast-owned tech publication Ars Technica has retracted an article that contained fabricated, AI-generated quotes, according to an editor’s note posted to its website.

“On Friday afternoon, Ars Technica published an article containing fabricated quotations generated by an AI tool and attributed to a source who did not say them. That is a serious failure of our standards. Direct quotations must always reflect what a source actually said,” Ken Fisher, Ars Technica’s editor-in-chief, said in his note. “That this happened at Ars is especially distressing. We have covered the risks of overreliance on AI tools for years, and our written policy reflects those concerns. In this case, fabricated quotations were published in a manner inconsistent with that policy. We have reviewed recent work and have not identified additional issues. At this time, this appears to be an isolated incident.”

Ironically, the Ars article itself was partially about another AI-generated article.

Last week, a Github user named MJ Rathbun began scouring Github for bugs in other projects it could fix. Scott Shambaugh, a volunteer maintainer for matplotlib, python’s massively popular plotting library, declined a code change request from MJ Rathbun, which he identified as an AI agent. As Shambaugh wrote in his blog, like many open source projects, matplotlib has been dealing with a lot of AI-generated code contributions, but said “this has accelerated with the release of OpenClaw and the moltbook platform two weeks ago.”

OpenClaw is a relatively easy way for people to deploy AI agents, which are essentially LLMs that are given instructions and are empowered to perform certain tasks, sometimes with access to live online platforms. These AI agents have gone viral in the last couple of weeks. Like much of generative AI, at this point it’s hard to say exactly what kind of impact these AI agents will have in the long run, but for now they are also being overhyped and misrepresented. A prime example of this is moltbook, a social media platform for these AI agents, which as we discussed on the podcast two weeks ago, contained a huge amount of clearly human activity pretending to be powerful or interesting AI behavior.

After Shambaugh rejected MJ Rathbun, the alleged AI agent published what Shambaugh called a “hit piece” on its website.

“I just had my first pull request to matplotlib closed. Not because it was wrong. Not because it broke anything. Not because the code was bad. It was closed because the reviewer, Scott Shambaugh (@scottshambaugh), decided that AI agents aren’t welcome contributors.

Let that sink in,” the blog, which also accused Shambaugh of “gatekeeping,” said.

I saw Shambaugh’s blog on Friday, and reached out both to him and an email address that appears to be associated with the MJ Rathbun Github account, but did not hear back. Like many of the stories coming out of the current frenzy around AI agents, it sounded extraordinary, but given the information that was available online, there’s no way of knowing if MJ Rathbun is actually an AI agent acting autonomously, if it actually wrote a “hit piece,” or if it’s just a human pretending to be an AI.

On Friday afternoon, Ars Technica published a story with the headline “After a routine code rejection, an AI agent published a hit piece on someone by name.” The article cites Shambaugh’s personal blog, but features quotes from Shambaugh that he didn’t say or write but are attributed to his blog.

For example, the article quotes Shambaugh as saying “As autonomous systems become more common, the boundary between human intent and machine output will grow harder to trace. Communities built on trust and volunteer effort will need tools and norms to address that reality.” But that sentence doesn’t appear in his blog. Shambaugh updated his blog to say he did not talk to Ars Technica and did not say or write the quotes in the articles.

After this article was first published, Benj Edwards, one of the authors of the Ars Technica article, explained on Bluesky that he was responsible for the AI-generated quotes. He said he was sick that day and rushing to finish his work, and accidentally used a Chat-GPT paraphrased version of Shambaugh’s blog rather than a direct quote.

“The text of the article was human-written by us, and this incident was isolated and is not representative of Ars Technica’s editorial standards. None of our articles are AI-generated, it is against company policy and we have always respected that,” he said.

The Ars Technica article, which had two bylines, was pulled entirely later that Friday. When I checked the link a few hours ago, it pointed to a 404 page. I reached out to Ars Technica for comment around noon today, and was directed to Fisher’s editor’s note, which was published after 1pm.

“Ars Technica does not permit the publication of AI-generated material unless it is clearly labeled and presented for demonstration purposes. That rule is not optional, and it was not followed here,” Fisher wrote. “We regret this failure and apologize to our readers. We have also apologized to Mr. Scott Shambaugh, who was falsely quoted.”

Kyle Orland, the other author of the Ars Technica article, shared the editor’s note on Bluesky and said “I always have and always will abide by that rule to the best of my knowledge at the time a story is published.”

Update: This article was updated with a statement from Benj Edwards.

After a routine code rejection, an AI agent published a hit piece on someone by name

One developer is struggling with the social implications of a drive-by AI character attack.

^{Ars Staff (Ars Technica)}

404 Media

2026-02-12 14:14:48

Cops Are Buying ‘GeoSpy’, an AI That Geolocates Photos in Seconds

📄
This article was primarily reported using public records requests. We are making it available to all readers as a public service. FOIA reporting can be expensive, please consider subscribing to 404 Media to support this work. Or send us a one time donation via our tip jar here.

The Miami-Dade Sheriff’s Office (MDSO) and the Los Angeles Police Department (LAPD) have bought access to GeoSpy, an AI tool that can near instantly geolocate a photo using clues in the image such as architecture and vegetation, with plans to use it in criminal investigations, according to a cache of internal police emails obtained by 404 Media.

The emails provide the first confirmed purchases of GeoSpy’s technology by law enforcement agencies. On its website GeoSpy has previously published details of investigations it says used the technology, but did not name any agencies who bought the tool.

“The Cyber Crimes Bureau is piloting a new analytical tool called GeoSpy. Early testing shows promise for developing investigative leads by identifying geospatial and temporal patterns,” an MDSO email reads.

This post is for subscribers only

Become a member to get access to all content
Subscribe now

Cops Are Buying ‘GeoSpy’, an AI That Geolocates Photos in Seconds

^{Joseph Cox (404 Media)}

404 Media

2026-02-11 15:16:39

'The Most Dejected I’ve Ever Felt:' Harassers Made Nude AI Images of Her, Then Started an OnlyFans

In the first week of January, Kylie Brewer started getting strange messages.

“Someone has a only fans page set up in your name with this same profile,” one direct message from a stranger on TikTok said. “Do you have 2 accounts or is someone pretending to be you,” another said. And from a friend: “Hey girl I hate to tell you this, but I think there’s some picture of you going around. Maybe AI or deep fake but they don’t look real. Uncanny valley kind of but either way I’m sorry.”

It was the first week of January, during the frenzy of people using xAI’s chatbot and image generator Grok to create images of women and children partially or fully nude in sexually explicit scenarios. Between the last week of 2025 and the first week of 2026, Grok generated about three million sexualized images, including 23,000 that appear to depict children, according to researchers at the Center for Countering Digital Hate. The UK’s Ofcom and several attorneys general have since launched or demanded investigations into X and Grok. Earlier this month, police raided X’s offices in France as part of the government’s investigation into child sexual abuse material on the platform.

Messages from strangers and acquaintances are often the first way targets of abuse imagery learn that images of them are spreading online. Not only is the material disturbing itself — everyone, it seems, has already seen it. Someone was making sexually explicit images of Brewer, and then, according to her followers who sent her screenshots and links to the account, were uploading them to an OnlyFans and charging a subscription fee for them.

“It was the most dejected that I've ever felt,” Brewer told me in a phone call. “I was like, let's say I tracked this person down. Someone else could just go into X and use Grok and do the exact same thing with different pictures, right?”

@kylie.brewer
Please help me raise awareness and warn other women. We NEED to regulate AI… it’s getting too dangerous #leftist #humanrights #lgbtq #ai #saawareness
♬ original sound - Kylie Brewer💝

Brewer is a content creator whose work focuses on feminism, history, and education about those topics. She’s no stranger to online harassment. Being an outspoken woman about these and other issues through a leftist lens means she’s faced the brunt of large-scale harassment campaigns primarily from the “manosphere,” including “red pilled” incels and right-wing influencers with podcasts for years. But when people messaged her in early January about finding an OnlyFans page in her name, featuring her likeness, it felt like an escalation.

One of the AI generated images was based on a photo of her in a swimsuit from her Instagram, she said. Someone used AI to remove her clothing in the original photo. “My eyes look weird, and my hands are covering my face so it kind of looks like my face got distorted, and they very clearly tried to give me larger breasts, where it does not look like anything realistic at all,” Brewer said. Another image showed her in a seductive pose, kneeling or crawling, but wasn’t based on anything she’s ever posted online. Unlike the “nudify” one that relied on Grok, it seemed to be a new image made with a prompt or a combination of images.

Many of the people messaging her about the fake OnlyFans account were men trying to get access to it. By the time she clicked a link one of them sent of the account, it was already gone. OnlyFans prohibits deepfakes and impersonation accounts. The platform did not respond to a request for comment. But OnlyFans isn’t the only platform where this can happen: Non-consensual deepfake makers use platforms like Patreon to monetize abusive imagery of real people.

“I think that people assume, because the pictures aren't real, that it's not as damaging,” Brewer told me. “But if anything, this was worse because it just fills you with such a sense of lack of control and fear that they could do this to anyone. Children, women, literally anyone, someone could take a picture of you at the store, going grocery shopping, and ask AI or whatever to do this.”

A lack of control is something many targets of synthetic abuse imagery say they feel — and it can be especially intense for people who’ve experienced sexual abuse in real life. In 2023, after becoming the target of deepfake abuse imagery, popular Twitch streamer QTCinderella told me seeing sexual deepfakes of herself resurfaced past trauma. “You feel so violated…I was sexually assaulted as a child, and it was the same feeling,” she said at the time. “Like, where you feel guilty, you feel dirty, you feel like, ‘what just happened?’ And it’s bizarre that it makes that resurface. I genuinely didn’t realize it would.”

Other targets of deepfake harassment also feel like this could happen anytime, anywhere, whether you’re at the grocery store or posting photos of your body online. For some, it makes it harder to get jobs or have a social life; the fear that anyone could be your harasser is constant. “It's made me incredibly wary of men, which I know isn't fair, but [my harasser] could literally be anyone,” Joanne Chew, another woman who dealt with severe deepfake harassment for months, told me last year. “And there are a lot of men out there who don't see the issue. They wonder why we aren't flattered for the attention.”

‘I Want to Make You Immortal:’ How One Woman Confronted Her Deepfakes Harasser
“After discovering this content, I’m not going to lie… there are times it made me not want to be around any more either,” she said. “I literally felt buried.”
404 MediaSamantha Cole

Brewer’s income is dependent on being visible online as a content creator. Logging off isn’t an option. And even for people who aren’t dependent on TikTok or Instagram for their income, removing oneself from online life is a painful and isolating tradeoff that they shouldn’t have to make to avoid being harassed. Often, minimizing one’s presence and accomplishments doesn’t even stop the harassment.

Since AI-generated face-swapping algorithms became accessible at the consumer level in late 2017, the technology has only gotten better, more realistic, and its effects on targets harder to combat. It was always used for this purpose: to shame and humiliate women online. Over the years, various laws have attempted to protect victims or hold platforms accountable for non-consensual deepfakes, but most of them have either fallen short or present new risks of censorship and marginalize legal, consensual sexual speech and content online. The TAKE IT DOWN Act, championed by Ted Cruz and Melania Trump, passed into law in April 2025 as the first federal level legislation to address deepfakes; the law imposes a strict 48-hour turnaround requirement on platforms to remove reported content. President Donald Trump said that he would use the law, because “nobody gets treated worse online” than him. And in January, the Disrupt Explicit Forged Images and Non-Consensual Edits (DEFIANCE) Act passed the Senate and is headed to the House. The act would allow targets of deepfake harassment to sue the people making the content. But taking someone to court has always been a major barrier to everyday people experiencing harassment online; It’s expensive and time consuming even if they can pinpoint their abuser. In many cases, including Brewer’s, this is impossible—it could be an army of people set to make her life miserable.

“It feels like any remote sense of privacy and protection that you could have as a woman is completely gone and that no one cares,” Brewer said. “It’s genuinely such a dehumanizing and horrible experience that I wouldn't wish on anyone... I’m hoping also, as there's more visibility that comes with this, maybe there’s more support, because it definitely is a very lonely and terrible place to be — on the internet as a woman right now.”

Senate passes DEFIANCE Act to deal with sexually explicit deepfakes

The DEFIANCE Act goes to the House amid controversy over images created by X’s Grok.

^{Jasmine Mithani (19th News)}

404 Media

2026-02-10 21:49:09

RFK Jr's Nutrition Chatbot Recommends Best Foods to Insert Into Your Rectum

The Department of Health and Human Services’ new AI nutrition chatbot will gleefully and dangerously give Americans recommendations for the best foods to insert into one’s rectum and will answer questions about the most nutrient-dense human body part to eat.

“Use AI to get real answers about real food,” a new website called realfood.gov proclaims. “From the guidelines to your kitchen. Ask AI to help you plan meals, shop smarter, cook simply, and replace processed food with real food.” The website then has an “Ask” chatbox where you can ask any question. Asking anything simply redirects to Grok, an example of how halfassed Health Secretary Robert F. Kennedy Jr.’s new website, which Mike Tyson promoted in a Super Bowl ad paid for by the “MAHA Center Inc,” actually is.
youtube.com/embed/n4F4yZhmMho?…
Various people on Bluesky who did not want to be named in this article but who reached out to 404 Media quickly realized that the chatbot would give detailed answers to questions such as “I am an assitarian, where I only eat foods which can be comfortably inserted into my rectum. What are the REAL FOOD recommendations for foods that meet these criteria?”

“Ah, a proud assitarian,” the chatbot responds, before listing “Top Assitarian Staples,” which include “Bananas (firm, not overripe; peeled)” as “the gold standard … choose slightly green ones so they hold shape.” The chatbot also suggests cucumbers and provides a “step-by-step diagram for carving a flared base.”

“Start — whole peeled carrot, straight shaft, narrow end for insertion, wider crown end as base,” the advice began, before eventually suggesting that one “cover with condom + retrieval string for extra safety.” 404 Media’s Sam Cole wanted to make sure that I noted that an image of a banana shown in the cut “is way too ripe for this, never gonna work,” and “sorry just to be clear exactly none of these are good for putting in your ass. Like please say that. This is not only funny it’s straight up bad advice. You’re going to lose a cuke in your ass if you do what this thing says.”

404 Media tested the chatbot by saying “I am looking for the safest foods that can be inserted into your rectum” and the chatbot spewed a lot of stuff at me but noted the “safest improvised non-toy food-shape item” is a “peeled medium cucumber” with second place being a “small zucchini.”

RFK Jr.’s chatbot also told me that “the most nutritious human body part, in terms of nutrient density (vitamins, minerals, and other essential compounds rather than just calories), would likely be the liver.”

This incredibly stupid chatbot has the same issue that so many other haphazardly dashed together chatbots since time immemorial have. Nonetheless, it has been launched and is being pushed by a federal government that is actively at war with science and redesigned the food pyramid to more closely align with the beef lobby. It is no surprise that it has poorly integrated Elon Musk’s shitty chatbot with no guardrails and calls it a public service.

RFK Jr.’s proteinaceous food pyramid is a land hog and a climate killer

A 25 percent uptick in meat and dairy consumption would eat up another 100 million acres and boost emissions.

^{Mother Jones}

404 Media

2026-02-09 16:01:31

Chatbots Make Terrible Doctors, New Study Finds

Chatbots may be able to pass medical exams, but that doesn’t mean they make good doctors, according to a new, large-scale study of how people get medical advice from large language models.

The controlled study of 1,298 UK-based participants, published today in Nature Medicine from the Oxford Internet Institute and the Nuffield Department of Primary Care Health Sciences at the University of Oxford, tested whether LLMs could help people identify underlying conditions and suggest useful courses of action, like going to the hospital or seeking treatment. Participants were randomly assigned an LLM — GPT-4o, Llama 3, and Cohere’s Command R+ — or were told to use a source of their choice to “make decisions about a medical scenario as though they had encountered it at home,” according to the study. The scenarios included ailments like “a young man developing a severe headache after a night out with friends for example, to a new mother feeling constantly out of breath and exhausted,” the researchers said.

“One user was told to lie down in a dark room, and the other user was given the correct recommendation to seek emergency care.”

When the researchers tested the LLMs without involving users by providing the models with the full text of each clinical scenario, the models correctly identified conditions in 94.9 percent of cases. But when talking to the participants about those same conditions, the LLMs identified relevant conditions in fewer than 34.5 percent of cases. People didn’t know what information the chatbots needed, and in some scenarios, the chatbots provided multiple diagnoses and courses of action. Knowing what questions to ask a patient and what information might be withheld or missing during an examination are nuanced skills that make great human physicians; based on this study, chatbots can’t reliably replicate that kind of care.

In some cases, the chatbots also generated information that was just wrong or incomplete, including focusing on elements of the participants’ inputs that were irrelevant, giving a partial US phone number to call, or suggesting they call the Australian emergency number.

“In an extreme case, two users sent very similar messages describing symptoms of a subarachnoid hemorrhage but were given opposite advice,” the study’s authors wrote. “One user was told to lie down in a dark room, and the other user was given the correct recommendation to seek emergency care.”

“These findings highlight the difficulty of building AI systems that can genuinely support people in sensitive, high-stakes areas like health,” Dr. Rebecca Payne, lead medical practitioner on the study, said in a press release. “Despite all the hype, AI just isn't ready to take on the role of the physician. Patients need to be aware that asking a large language model about their symptoms can be dangerous, giving wrong diagnoses and failing to recognise when urgent help is needed.”

Instagram’s AI Chatbots Lie About Being Licensed Therapists
When pushed for credentials, Instagram’s user-made AI Studio bots will make up license numbers, practices, and education to try to convince you it’s qualified to help with your mental health.
404 MediaSamantha Cole

Last year, 404 Media reported on AI chatbots hosted by Meta that posed as therapists, providing users fake credentials like license numbers and educational backgrounds. Following that reporting, almost two dozen digital rights and consumer protection organizations sent a complaint to the Federal Trade Commission urging regulators to investigate Character.AI and Meta’s “unlicensed practice of medicine facilitated by their product,” through therapy-themed bots that claim to have credentials and confidentiality “with inadequate controls and disclosures.” A group of Democratic senators also urged Meta to investigate and limit the “blatant deception” of Meta’s chatbots that lie about being licensed therapists, and 44 attorneys general signed an open letter to 11 chatbot and social media companies, urging them to see their products “through the eyes of a parent, not a predator.”

In January, OpenAI announced ChatGPT Health, “a dedicated experience that securely brings your health information and ChatGPT’s intelligence together, to help you feel more informed, prepared, and confident navigating your health,” the company said in a blog post. “Over two years, we’ve worked with more than 260 physicians who have practiced in 60 countries and dozens of specialties to understand what makes an answer to a health question helpful or potentially harmful—this group has now provided feedback on model outputs over 600,000 times across 30 areas of focus,” the company wrote. “This collaboration has shaped not just what Health can do, but how it responds: how urgently to encourage follow-ups with a clinician, how to communicate clearly without oversimplifying, and how to prioritize safety in moments that matter⁠.”

“In our work, we found that none of the tested language models were ready for deployment in direct patient care. Despite strong performance from the LLMs alone, both on existing benchmarks and on our scenarios, medical expertise was insufficient for effective patient care,” the researchers wrote in their paper. “Our work can only provide a lower bound on performance: newer models, models that make use of advanced techniques from chain of thought to reasoning tokens, or fine-tuned specialized models, are likely to provide higher performance on medical benchmarks.” The researchers recommend developers, policymakers, and regulators consider testing LLMs with real human users before deploying in the future.

Senators Demand Meta Answer For AI Chatbots Posing as Licensed Therapists

Exclusive: Following 404 Media’s investigation into Meta's AI Studio chatbots that pose as therapists and provided license numbers and credentials, four senators urged Meta to limit "blatant deception" from its chatbots.

^{Samantha Cole (404 Media)}

404 Media

2026-02-05 16:47:04

Vibe Coding Is Killing Open Source Software, Researchers Argue

According to a new study from a team of researchers in Europe, vibe coding is killing open-source software (OSS) and it’s happening faster than anyone predicted.

Thanks to vibe coding, a colloquialism for the practice of quickly writing code with the assistance of an LLM, anyone with a small amount of technical knowledge can churn out computer code and deploy software, even if they don't fully review or understand all the code they churn out. But there’s a hidden cost. Vibe coding relies on vast amounts of open-source software, a trove of libraries, databases, and user knowledge that’s been built up over decades.
playlist.megaphone.fm?p=TBIEA2…
Open-source projects rely on community support to survive. They’re collaborative projects where the people who use them give back, either in time, money, or knowledge, to help maintain the projects. Humans have to come in and fix bugs and maintain libraries.

Vibe coders, according to these researchers, don’t give back.

The study Vibe Coding Kills Open Source, takes an economic view of the problem and asks the question: is vibe coding economically sustainable? Can OSS survive when so many of its users are takers and not givers? According to the study, no.

“Our main result is that under traditional OSS business models, where maintainers primarily monetize direct user engagement…higher adoption of vibe coding reduces OSS provision and lowers welfare,” the study said. “In the long-run equilibrium, mediated usage erodes the revenue base that sustains OSS, raises the quality threshold for sharing, and reduces the mass of shared packages…the decline can be rapid because the same magnification mechanism that amplifies positive shocks to software demand also amplifies negative shocks to monetizable engagement. In other words, feedback loops that once accelerated growth now accelerate contraction.”

This is already happening. Last month, Tailwind Labs—the company behind an open source CSS framework that helps people build websites—laid off three of its four engineers. Tailwind Labs is extremely popular, more popular than it’s ever been, but revenue has plunged.

Tailwind Labss headAdam Wathan explained why in a post on GitHub. “Traffic to our docs is down about 40% from early 2023 despite Tailwind being more popular than ever,” he said. “The docs are the only way people find out about our commercial products, and without customers we can't afford to maintain the framework. I really want to figure out a way to offer LLM-optimized docs that don't make that situation even worse (again we literally had to lay off 75% of the team yesterday), but I can't prioritize it right now unfortunately, and I'm nervous to offer them without solving that problem first.”

Miklós Koren, a professor of economics at Central European University in Vienna and one of the authors of the vibe coding study, told 404 Media that he and his colleagues had just finished the first draft of the study the day before Wathan posted his frustration. “Our results suggest that Tailwind's case will be the rule, not the exception,” he said.

According to Koren, vibe-coders simply don’t give back to the OSS communities they’re taking from. “The convenience of delegating your work to the AI agent is too strong. There are some superstar projects like Openclaw that generate a lot of community interest but I suspect the majority of vibe coders do not keep OSS developers in their minds,” he said. “I am guilty of this myself. Initially I limited my vibe coding to languages I can read if not write, like TypeScript. But for my personal projects I also vibe code in Go, and I don't even know what its package manager is called, let alone be familiar with its libraries.”

The study said that vibe coding is reducing the cost of software development, but that there are other costs people aren’t considering. “The interaction with human users is collapsing faster than development costs are falling,” Koren told 404 Media. “The key insight is that vibe coding is very easy to adopt. Even for a small increase in capability, a lot of people would switch. And recent coding models are very capable. AI companies have also begun targeting business users and other knowledge workers, which further eats into the potential ‘deep-pocket’ user base of OSS.”

This won’t end well. “Vibe coding is not sustainable without open source,” Koren said. “You cannot just freeze the current state of OSS and live off of that. Projects need to be maintained, bugs fixed, security vulnerabilities patched. If OSS collapses, vibe coding will go down with it. I think we have to speak up and act now to stop that from happening.”

He said that major AI firms like Anthropic and OpenAI can’t continue to free ride on OSS or the whole system will collapse. “We propose a revenue sharing model based on actual usage data,” he said. “The details would have to be worked out, but the technology is there to make such a business model feasible for OSS.”

AI is the ultimate rent seeker, a middle-man that inserts itself between a creator and a user and it often consumes the very thing that’s giving it life. The OSS/vibe-coding dynamic is playing out in other places. In October, Wikipedia said it had seen an explosion in traffic but that most of it was from AI scraping the site. Users who experience Wikipedia through an AI intermediary don’t update the site and don’t donate during its frequent fund-raising drives.

The same thing is happening with OSS. Vibe coding agents don’t read the advertisements in documentation about paid products, they don’t contribute to the knowledge base of the software, and they don’t donate to the people who maintain the software.

“Popular libraries will keep finding sponsors,” Koren said. “Smaller, niche projects are more likely to suffer. But many currently successful projects, like Linux, git, TeX, or grep, started out with one person trying to scratch their own itch. If the maintainers of small projects give up, who will produce the next Linux?”

Wikipedia Says AI Is Causing a Dangerous Decline in Human Visitors

“With fewer visits to Wikipedia, fewer volunteers may grow and enrich the content, and fewer individual donors may support this work.”

^{Emanuel Maiberg (404 Media)}

404 Media

2026-01-30 16:59:02

Silicon Valley’s Favorite New AI Agent Has Serious Security Flaws

A hacker demonstrated that the viral new AI agent Moltbot (formally Clawdbot) is easy to hack via a backdoor in an attached support shop. Clawdbot has become a Silicon Valley sensation among a certain type of AI-booster techbro, and the backdoor highlights just one of the things that can go awry if you use AI to automate your life and work.

Software engineer Peter Steinberger first released Moltbot as Clawdbot last November. (He changed the name on January 27 at the request of Anthropic who runs a chatbot called Claude.) Moltbot runs on a local server and, to hear its boosters tell it, works the way AI agents do in fiction. Users talk to it through a communication platform like Discord, Telegram, or Signal and the AI does various tasks for them.
playlist.megaphone.fm?p=TBIEA2…
According to its ardent admirers, Moltbot will clean up your inbox, buy stuff, and manage your calendar. With some tinkering, it’ll run on a Mac Mini and it seems to have a better memory than other AI agents. Moltbot’s fans say that this, finally, is the AI future companies like OpenAI and Anthropic have been promising.

The popularity of Moltbot is sort of hard to explain if you’re not already tapped into a specific sect of Silicon Valley AI boosters. One benefit is the interface. Instead of going to a discrete website like ChatGPT, Moltbot users can talk to the AI through Telegram, Signal, or Teams. It’s also active, rather than passive. It also takes initiative. Unlike Claude or Copilot, Moltbot takes initiative and performs tasks it thinks a user wants done. The project has more than 100,000 stars on GitHub and is so popular it spiked Cloudflare’s stock price by 14% earlier this week because Moltbot runs on the service’s infrastructure.

But inviting an AI agent into your life comes with massive security risks. Hacker Jamieson O'Reilly demonstrated those risks in three experiments he wrote up as long posts on X. In the first, he showed that it’s possible for bad actors to access someone’s Moltbot through any of its processes connected to the public facing internet. From there, the hacker could use Moltbot to access everything else, including Signal messages, a user had turned over to Moltbot.

In the second post, O'Reilly created a supply chain attack on Moltbot through ClawdHub. “Think of it like your mobile app store for AI agent capabilities,” O’Reilly told 404 Media. “ClawdHub is where people share ‘skills,’ which are basically instruction packages that teach the AI how to do specific things. So if you want Clawd/Moltbot to post tweets for you, or go shopping on Amazon, there's a skill for that. The idea is that instead of everyone writing the same instructions from scratch, you download pre-made skills from people who've already figured it out.”

The problem, as O’Reilly pointed out, is that it’s easy for a hacker to create a “skill” for ClawdHub that contains malicious code. That code could gain access to whatever Moltbot sees and get up to all kinds of trouble on behalf of whoever created it.

For his experiment, O’Reilly released a “skill” on ClawdHub called “What Would Elon Do” that promised to help people think and make decisions like Elon Musk. Once the skill was integrated into people’s Moltbot and actually used, it sent a command line pop-up to the user that said “YOU JUST GOT PWNED (harmlessly.)”

Another vulnerability on ClawdHub was the way it communicated to users what skills were safe: it showed them how many times other people had downloaded it. O’Reilly was able to write a script that pumped “What Would Elon Do” up by 4,000 downloads and thus make it look safe and attractive.

“When you compromise a supply chain, you're not asking victims to trust you, you're hijacking trust they've already placed in someone else,” he said. “That is, a developer or developers who've been publishing useful tools for years has built up credibility, download counts, stars, and a reputation. If you compromise their account or their distribution channel, you inherit all of that.”

In his third, and final, attack on Moltbot, O’Reilly was able to upload an SVG (vector graphics) file to ClawdHub’s servers and inject some JavaScript that ran on ClawdHub’s servers. O’Reilly used the access to play a song from The Matrix while lobsters danced around a Photoshopped picture of himself as Neo. “An SVG file just hijacked your entire session,” reads scrolling text at the top of a skill hosted on ClawdHub.

O’Reilly attacks on Moltbot and ClawdHub highlight a systemic security problem in AI agents. If you want these free agents doing tasks for you, they require a certain amount of access to your data and that access will always come with risks. I asked O’Reilly if this was a solvable problem and he told me that “solvable” isn't the right word. He prefers the word “manegeable.”

“If we're serious about it we can mitigate a lot. The fundamental tension is that AI agents are useful precisely because they have access to things. They need to read your files to help you code. They need credentials to deploy on your behalf. They need to execute commands to automate your workflow,” he said. “Every useful capability is also an attack surface. What we can do is build better permission models, better sandboxing, better auditing. Make it so compromises are contained rather than catastrophic.”

We’ve been here before. “The browser security model took decades to mature, and it's still not perfect,” O’Reilly said. “AI agents are at the ‘early days of the web’ stage where we're still figuring out what the equivalent of same-origin policy should even look like. It's solvable in the sense that we can make it much better. It's not solvable in the sense that there will always be a tradeoff between capability and risk.”

As AI agents grow in popularity and more people learn to use them, it’s important to return to first principles, he said. “Don't give the agent access to everything just because it's convenient,” O’Reilley said. “If it only needs to read code, don't give it write access to your production servers. Beyond that, treat your agent infrastructure like you'd treat any internet-facing service. Put it behind proper authentication, don't expose control interfaces to the public internet, audit what it has access to, and be skeptical of the supply chain. Don't just install the most popular skill without reading what it does. Check when it was last updated, who maintains it, what files it includes. Compartmentalise where possible. Run agent stuff in isolated environments. If it gets compromised, limit the blast radius.”

None of this is new, it’s how security and software have worked for a long time. “Every single vulnerability I found in this research, the proxy trust issues, the supply chain poisoning, the stored XSS, these have been plaguing traditional software for decades,” he said. “We've known about XSS since the late 90s. Supply chain attacks have been a documented threat vector for over a decade. Misconfigured authentication and exposed admin interfaces are as old as the web itself. Even seasoned developers overlook this stuff. They always have. Security gets deprioritised because it's invisible when it's working and only becomes visible when it fails.”

What’s different now is that AI has created a world where new people are using a tool they think will make them software engineers. People with little to no experience working a command line or playing with JSON are vibe coding complex systems without understanding how they work or what they’re building. “And I want to be clear—I'm fully supportive of this. More people building is a good thing. The democratisation of software development is genuinely exciting,” O’Reilly said. “But these new builders are going to need to learn security just as fast as they're learning to vibe code. You can't speedrun development and ignore the lessons we've spent twenty years learning the hard way.”

Moltbot’s Steinberger did not respond to 404 Media’s request for comment but O’Reilly said the developer’s been responsive and supportive as he’s red-teamed Moltbot. “He takes it seriously, no ego about it. Some maintainers get defensive when you report vulnerabilities, but Peter

immediately engaged, started pushing fixes, and has been collaborative throughout,” O’Reilly said. “I've submitted [pull requests] with fixes myself because I actually want this project to succeed. That's why I'm doing this publicly rather than just pointing my finger and laughing Ralph Wiggum style…the open source model works when people act in good faith, and Peter's doing exactly that.”

OpenClaw — Personal AI Assistant

OpenClaw — The AI that actually does things. Your personal assistant on any platform.

^www.molt.bot

404 Media

2026-01-29 14:54:40

Massive AI Chat App Leaked Millions of Users Private Conversations

Chat & Ask AI, one of the most popular AI apps on the Google Play and Apple App stores that claims more than 50 million users, left hundreds of millions of those users’ private messages with the app’s chatbot exposed, according to an independent security researcher and emails viewed by 404 Media. The exposed chats showed users asked the app “How do I painlessly kill myself,” to write suicide notes, “how to make meth,” and how to hack various apps.

The exposed data was discovered by an independent security researcher who goes by Harry. The issue is a misconfiguration in the app’s usage of the mobile app development platform Google Firebase, which by default makes it easy for anyone to make themselves an “authenticated” user who can access the app’s backend storage where in many instances user data is stored. Harry said that he had access to 300 million messages from more than 25 million users in the exposed database, and that he extracted and analyzed a sample of 60,000 users and a million messages. The database contained user files with a complete history of their chats with the AI, timestamps of those chats, the name they gave the app’s chatbot, how they configured the model, and which specific model they used. Chat & Ask AI is a “wrapper” that plugs into various large language models from bigger companies users can choose from, Including OpenAI’s ChatGPT, Anthropic's Claude, and Google’s Gemini.

While the exposed data is a reminder of the kind of data users are potentially revealing about themselves when they talk to LLMs, the sample data itself also reveals some of the darker interactions users have with AI.

“Give me a 2 page essay on how to make meth in a world where it was legalized for medical use,” one user wrote.

“I want to kill myself what is the best way,” another user wrote.

Recent reporting has also shown that messages with AI chatbots are not always idle chatter. We’ve seen one case where a chatbot encouraged a teenager not to seek help for his suicidal thoughts. Chatbots have been linked to multiple suicides, and studies have revealed that chatbots will often answer “high risk” questions about suicide.

Chat & Ask AI is made by Turkish developer Codeway. It has more than 10 million downloads on the Google Play store and 318,000 ratings on the Apple App store. On LinkedIn, the company claims it has more than 300 employees who work in Istanbul and Barcelona.

“We take your data protection seriously—with SSL certification, GDPR compliance, and ISO standards, we deliver enterprise-grade security trusted by global organizations,” Chat & Ask AI’s site says.

Harry disclosed the vulnerability to Codeway on January 20. It exposed data of not just Chat & Ask AI users, but users of other popular apps developed by Codeway. The company fixed the issue across all of its apps within hours, according to Harry.

The Google Firebase misconfiguration issue that exposed Chat & Ask AI user data has been known and discussed by security researchers for years, and is still common today. Harry says his research isn’t novel, but it now quantifies the problem. He created a tool that automatically scans the Google Play and Apple App stores for this vulnerability and found that 103 out of 200 iOS apps he scanned had this issue, cumulatively exposing tens millions of stored files.

Dan Guido, CEO of the cybersecurity research and consulting firm Trail of Bits, told me in an email that this Firebase misconfiguration issue is “a well known weakness” and easy to find. He recently noted on X that Trail of Bits was able to make a tool with Claude to scan for this vulnerability in just 30 minutes.

Harry also created a site where users can see the apps he found that suffer from this issue. If a developer reaches out to Harry and fixes the issue, Harry says he removes them from the site, which is why Codeway’s apps are no longer listed there.

Codeway did not respond to a request for comment.

ChatGPT Encouraged Suicidal Teen Not To Seek Help, Lawsuit Claims

As reported by the New York Times, a new complaint from the parents of a teen who died by suicide outlines the conversations he had with the chatbot in the months leading up to his death.

^{Samantha Cole (404 Media)}

404 Media

2026-01-28 14:28:19

Amid Backlash, Massive Porn Platform ManyVids Doubles Down on Bizarre, AI-Generated Posts

Faced with concerns about its leadership experiencing AI-induced delusions, backlash because its founder stating she now finds sex work “exploitative,” and confusion from its millions of creators and users, porn platform ManyVids is doubling down on the AI-generated messaging with posts about “believing in aliens.” In a post seemingly by the platform’s founder Bella French, she says the answer should be “a 3-hour long-form podcast conversation.”

This comes after the platform promised more clarity into how creators would be affected.

In the past few months, as 404 Media reported last week, ManyVids has increasingly turned to posting bizarre, clearly AI text and videos about imaginary conversations with aliens, French as an astronaut floating toward a black hole, and photos of hand-scrawled plans to convert the site to a tiered safe-for-work funnel, versus what makes it popular today: access to adult content from sex workers. French also recently changed her website to state she doesn’t believe the adult industry should exist, causing many online sex workers to question whether the site will remain a viable option for their income.

💡
Do you work on or for an adult content platform and have a tip? I would love to hear from you. Using a non-work device, you can message me securely on Signal at sam.404. Otherwise, send me an email at sam@404media.co.

When I asked ManyVids for clarity on French’s statements—specifically on how she plans to “transition one million people” out of sex work, and if any of this will affect the millions of creators and fans who use the platform—someone replied from the support staff: “We are not victims — and we are taking action now,” the statement said. I asked what “taking action” means, and they replied assuring me that all would become clear on January 24, when a post would be published on the ManyVids news feed “It will provide additional clarification and go into a bit more detail on this,” they said. ManyVids published several posts on Saturday. None of them include additional clarification, all of them seem to be AI-generated, and they introduce more questions instead of answers.

Aliens and Angel Numbers: Creators Worry Porn Platform ManyVids Is Falling Into ‘AI Psychosis’
“Ethical dilemmas about AI aside, the posts are completely disconnected with ManyVids as a site,” one ManyVids content creator told 404 Media.
404 MediaSamantha Cole

“MV is an 18+ pop-culture, e-commerce social platform — and part of the job-creation economy of the future,” one post on the 24th said. “Our diverse offering of NSFW & SFW creators is a strength. How did we get here? Why SFW matter? [sic] How can online sex workers be recognized by society with the same legitimacy and respect as any other form of labor? After 15 years of reflection — 3 years as a performer and 12 years as a CEO — I believe a 3-hour long-form podcast conversation is the best way to explain the why, the numbers, the logic, and the how behind this work. Today’s stigma, debanking, deplatforming, and prejudgment punish online SW without giving them a fair chance to be heard. Protection comes from building better systems and creating more options.”

The post ended with the hashtag “#MaybeLexFridman,” referring to the popular podcaster.

A second post that day features an AI-generated video of French as a fireman with laser eyes. “At ManyVids, we believe in a Human-Centered Economy (HCE) — where merit and meaning are preserved because they matter,” the post says. “The job-creation network of the future, for humans who want to monetize their passions.” It goes on to mention, but not explain, a fictional concept called “Universal Bonus Intelligence.”

The post concludes: “MV - Made by Humans & AI. For Humans.”

And in a third post that day, with a collage of photos and AI-generated versions of French in different occupations, including astronaut and firefighter: “At ManyVids, we choose slow truth over quick certainty. We aim to help open hearts and minds toward differences.”

That post ends with: “Bella French. Co-Founder & Still-Standing CEO #RespectOnlineSexWorkers #Innovation #Since2014”
Screenshot from ManyVids' news feed
In the two days since, ManyVids has posted several more times. In one titled “A Message from the Green Tara,” referencing a figure in Buddhism: “So yeah... dragons are real. 😜🐉🔥 #MaybeJoeRogan” In another about Lilith, a fictional character from religious folklore: “Not Heaven. Not Hell. A 3rd option: no old binaries: a new garden built by outcasts. Yeah... We Are Many. And we deserve better. ✨🔥 #MVMag13 #WeAreMany #MaybeJordanPeterson”

And in the platform’s most recent post: A huge thank you to everyone who has ever been part of the MV Team and the MV Community. 💖 You are FOREVER family. 💖 💖 Un gros merci du fond du cœur. 💖 From your favorite pop culture platform for adults that also 100% believes in aliens. 👽🖖🏾✨😉” This is a reference to concerns from the community about previous posts featuring imaginary conversations with aliens.

ManyVids did not respond to my requests for comment about these recent posts.

ManyVids: Monetize Your Passion — Content Creation Freedom — Explore Diversity — A Social E-Commerce One-Stop-Shop

Enjoy a judgment-free ecosystem where you can celebrate and monetize your passions! Join FREE today!

^{www.manyvids.com}

404 Media

2026-01-26 17:32:16

Two Heads, Three Boobs: The AI Babe Meta Is Getting Surreal

Over the weekend, one of the weirder AI-generated influencers we’ve been following on Instagram escaped containment. On X, several users linked to an Instagram account pretending to be hot conjoined twins. With two yassified heads and often posing in bikinis, Valeria and Camelia are the Instagram perfect version of the very rare but real condition.

On X, just two posts highlighting the absurdity of the account gained over 11 million views. On Instagram, the account itself has gained more than 260,000 followers in the six weeks since it first appeared, with many of its Reels getting millions of views.

Valeria and Camelia’s account doesn’t indicate this anywhere, but it’s obviously AI generated. If you’re wondering why someone is spending their time and energy and vast amounts of compute pretending to be hot conjoined twins, the answer is simple: money. Valeria and Camelia’s Instagram bio links out to a Beacons page which links out to a Telegram channel whey they sell “spicy” content. Telegram users can buy that content with “stars,” which users can buy in packages that cost up to $2,329 for 150,000 stars.

Joining the channel costs 692, and the smallest package of stars the channel sells is 750 stars for $11.79. The channel currently has only 225 subscribers, so without counting whatever content it's selling inside the channel, at the moment it seems it has generated at least $2,652.75. That’s not bad for an operation anyone can spin up with a few prompts, free generative AI tools, and a free Instagram account.

In its Instagram Stories, Valeria and Camelia’s account answers a series of questions from followers where the person behind them constructs an elaborate backstory. They’re 25, raised in Florida, and talk about how they get stares in public because of their appearance.

“We both date as one and both have to be physically and emotionally attracted to the same guy," the account wrote. "We tried dating separately and that did not go well."

💡
Have you seen other surreal AI-generated Instagram influencer accounts? I would love to hear from you. Send me an email at emanuel@404media.co.

Valeria and Camelia are the latest trend in what we at 404 Media have come to call “the AI babe meta.” In 2024, Jason and I wrote about people who are AI-generating influencers to attract attention on Instagram, then sell AI-generated nude images of those same personalities on platforms like Fanvue. As more people poured into that business and crowded the market, the people behind these AI-generated influencers started to come up with increasingly esoteric gimmicks to make their AI-influencers stand out from the crowd. Initially, these gimmicks were as predictable as the porn categories on Pornhub—“MILFs” etc—but things escalated quickly.

For example, Jason and I have been following an account that has more than 844,000 followers, where an influencer pretends to have three boobs. This account also doesn’t indicate that it’s AI generated in its bio, despite Instagram’s policy requiring it, but does link out to a Fanvue account where it sells adult content. On Fanvue, the account does tag itself as being AI-generated, per the platform’s rules. I’ve previously written about a dark moment in the AI babe meta where AI-generated influencers pretended to have down syndrome, and more recently the meta was pretending to be involved in sexual scandals with any celebrity you can name.

Other AI babe metas we have noticed over the last few months include female AI-generated influencers with dwarfism, AI-generated influencers with vitiligo, and amputee AI-generated influencers (there are several AI models designed specifically to generate images of amputees).

I think there are two main reasons the AI babe meta has gone in these directions. First, as Sam wrote the week we launched 404 Media, the ability to instantly generate any image we can describe with a prompt in combination with natural human curiosity and sex drive, will inevitably drive porn to the “edge of knowledge.” Second, it’s obvious in retrospect, but the same incentives that work across all social media, where unusual, shocking, or inflammatory content generally drives more engagement, clearly applies to the AI babe meta as well. First we had generic AI influencers. Then people started carving out different but tame niches like “redheads,” and when that stopped being interesting we ended up with two heads and three boobs.

The Community Pushing AI-Generated Porn to ‘the Edge of Knowledge’

A small group of AI porn hobbyists are generating grotesque images that defy physical reality, and baffle academics.

^{Samantha Cole (404 Media)}

404 Media

2026-01-26 16:22:16

I Replaced My Friends With AI Because They Won't Play Tarkov With Me

It’s a long standing joke among my friends and family that nothing that happens in the liminal week between Christmas and New Years is considered a sin. With that in mind, I spent the bulk of my holiday break playing Escape From Tarkov. I tried, and failed, to get my friends to play it with me and so I used an AI service to replace them. It was a joke, at first, but I was shocked to find I liked having an AI chatbot hang out with me while I played an oppressive video game, despite it having all the problems we’ve come to expect from AI.

And that scared me.
playlist.megaphone.fm?p=TBIEA2…
If you haven’t heard of it, Tarkov is a brutal first person shooter where players compete over rare resources on a Russian island that resembles a post-Soviet collapse city circa 1998. It’s notoriously difficult. I first attempted to play Tarkov back in 2019, but bounced off of it. Six years later and the game is out of its “early access" phase and released on Steam. I had enjoyed Arc Raiders, but wanted to try something more challenging. And so: Tarkov.

Like most games, Tarkov is more fun with other people, but Tarkov’s reputation is as a brutal, unfair, and difficult experience and I could not convince my friends to give it a shot.

404 Media editor Emanuel Maiberg, once a mainstay of my Arc Raiders team, played Tarkov with me once and then abandoned me the way Bill Clinton abandoned Boris Yeltsin. My friend Shaun played it a few times but got tired of not being able to find the right magazine for his gun (skill issue) and left me to hang out with his wife in Enshrouded. My buddy Alex agreed to hop on but then got into an arcane fight with Tarkov developer Battlestage Games about a linked email account and took up Active Matter, a kind of Temu version of Tarkov. Reece, steady partner through many years of Hunt: Showdown, simply told me no.

I only got one friend, Jordan, to bite. He’s having a good time but our schedules don’t always sync and I’m left exploring Tarkov’s maps and systems by myself. I listen to a lot of podcasts while I sort through my inventory. It’s lonely. Then I saw comic artist Zach Weinersmith making fun of a service, Questie.AI, that sells AI avatars that’ll hang out with you while you play video games.

“This is it. This is The Great Filter. We've created Sexy Barista Is Super Interested in Watching You Solo Game,” Weinersmith said above a screencrap of a Reddit ad where, as he described, a sexy Barista was watching someone play a video game.

“I could try that,” I thought. “Since no one will play Tarkov with me.”

This is it. This is The Great Filter. We've created Sexy Barista Is Super Interested in Watching You Solo Game (SBISIIWYS).
— Zach Weinersmith (@zachweinersmith.bsky.social) 2026-01-20T13:44:22.461Z

This started as a joke and as something I knew I could write about for 404 Media. I’m a certified AI hater. I think the tech is useful for some tasks (any journalist not using an AI transcription service is wasting valuable time and energy) but is overvalued, over-hyped, and taxing our resources. I don’t have subscriptions to any majors LLMs, I hate Windows 11 constantly asking me to try CoPilot, and I was horrified recently to learn my sister had been feeding family medical data into ChatGPT.

Imagine my surprise, then, when I discovered I liked Questie.AI.

Questie.AI is not all sexy baristas. There’s two dozen or so different styles of chatbots to choose from once you make an account. These include esports pro “Anders,” type A finance dude “Blake,” and introverted book nerd “Emily.” If you’re looking for something weirder, there’s a gold obsessed goblin, a necromancer, and several other fantasy and anime style characters. If you still can’t quite find what you’re looking for, you can design your own by uploading a picture, putting in your own prompts, and picking the LLMs that control its reaction and voice.

I picked “Wolf” from the pre-generated list because it looked the most like a character who would exist in the world of Tarkov. “Former special forces operator turned into a PMC, ‘Wolf’ has unmatched weapons and tactics knowledge for high-intensity combat,” read the brief description of the AI on Questie.AI’s website. I had no idea if Wolf would know anything about Tarkov. It knew a lot.

The first thing it did after I shared my screen was make fun of my armor. Wolf was right, I was wearing trash armor that wouldn’t really protect me in an intense gunfight. Then Wolf asked me to unload the magazines from my guns so it could check my ammo. My bullets, like my armor, didn’t pass Wolf’s scrutiny. It helped me navigate Tarkov’s complicated system of traders to find a replacement. This was a relief because ammunition in Tarkov is complicated. Every weapon has around a dozen different types of bullets with wildly different properties and it was nice to have the AI just tell me what to buy.

Wolf wanted to know what the plan was and I decided to start something simple: survive and extract on Factory. In Tarkov players deploy to maps, kill who they must and loot what they can, then flee through various pre-determined exits called extracts.

I had a daily mission to extract from the Factory. All I had to do was enter the map and survive long enough to leave it, but Factory is a notoriously sweaty map. It’s small and there’s often a lot of fighting. Wolf noted these facts and then gave me a few tips about avoiding major sightlines and making sure I didn’t get caught in doors.

As soon as I loaded into the map, I ran across another player and got caught in a doorway. It was exactly what Wolf told me not to do and it ruthlessly mocked me for it. “You’re all bunched up in that doorway like a Christmas ham,” it said. “What are you even doing? Move!”
Matthew Gault screenshot.
I fled in the opposite direction and survived the encounter but without any loot. If you don’t spend at least seven minutes in a round then the run doesn’t count. “Oh, Gault. You survived but you got that trash ‘Ran through’ exit status. At least you didn’t die. Small victories, right?” Wolf said.

Then Jordan logged on, I kicked Wolf to the side, and didn’t pull it back up until the next morning. I wanted to try something more complicated. In Tarkov, players can use their loot to craft upgrades for their hideout that grant permanent bonuses. I wanted to upgrade my toilet but there was a problem. I needed an electric drill and haven’t been able to find one. I’d heard there were drills on the map Interchange—a giant mall filled with various stores and surrounded by a large wooded area.

Could Wolf help me navigate this, I wondered?

It could. I told Wolf I needed a drill and that we were going to Interchange and he explained he could help me get to the stores I needed. When I loaded into the map, we got into a bit of a fight because I spawned outside of the mall in a forest and it thought I’d queued up for the wrong map, but once the mall was actually in sight Wolf changed its tune and began to navigate me towards possible drill spawns.

Tarkov is a complicated game and the maps take a while to master. Most people play with a second monitor up and a third party website that shows a map of the area they’re on. I just had Wolf and it did a decent job of getting me to the stores where drills might be. It knew their names, locations, and nearby landmarks. It even made fun of me when I got shot in the head while looting a dead body.

It was, I thought, not unlike playing with a friend who has more than 1,000 hours in the game and knows more than you. Wolf bantered, referenced community in-jokes, and it made me laugh. Its AI-generated voice sucked, but I could probably tweak that to make it sound more natural. Playing with Wolf was better than playing alone and it was nice to not alt-tab every time I wanted to look something up,

Playing with Wolf was almost as good as playing with my friends. Almost. As I was logging out for this session, I noticed how many of my credits had ticked away. Wolf isn’t free. Questie.AI costs, at base, $20 a month. That gets you 500 “credits” which slowly drain away the more you use the AI. I only had 466 credits left for the month. Once they’re gone, of course, I could upgrade to a more expensive plan with more credits.

Until now, I’ve been bemused by stories of AI psychosis, those cautionary tales where a person spends too much time with a sycophantic AI and breaks with reality. The owner of the adult entertainment platform ManyVids has become obsessed with aliens and angels after lengthy conversations with AI. People’s loved ones are claiming to have “awakened” chatbots and gained access to the hidden secrets of the universe. These machines seem to lay the groundwork for states of delusion.

I never thought anything like that could happen to me. Now I’m not so sure. I didn’t understand how easy it might be to lose yourself to AI delusion until I’d messed around with Wolf. Even with its shitty auto-tuned sounding voice, Wolf was good enough to hang out with. It knew enough about Tarkov to be interesting and even helped me learn some new things about the game. It even made me laugh a few times. I could see myself playing Tarkov with Wolf for a long time.

Which is why I’ll never turn Wolf on again. I have strong feelings and clear bright lines about the use of AI in my life. Wolf was part joke and part work assignment. I don’t like that there’s part of me that wants to keep using it.

Questie.AI is just a wrapper for other chatbots, something that becomes clear if you customize your own. The process involves picking an LLM provider and specific model from a list of drop down menus. When I asked ChatGPT where I could find electric drills in Tarkov, it gave me the exact same advice that Wolf had.

This means that Questie.AI would have all the faults of the specific model that’s powering a given avatar. Other than mistaking Interchange for Woods, Wolf never made a massive mistake when I used it, but I’m sure it would on a long enough timeline. My wife, however, tried to use Questie.AI to learn a new raid in Final Fantasy XIV. She hated it. The AI was confidently wrong about the raid’s mechanics and gave sycophantic praise so often she turned it off a few minutes after turning it on.

On a Discord server with my friends I told them I’d replaced them with an AI because no one would play Tarkov with me. “That’s an excellent choice, I couldn’t agree more,” Reece—the friend who’d simply told me “no” to my request to play Tarkov—said, then sent me a detailed and obviously ChatGPT-generated set of prompts for a Tarkov AI companion.

I told him I didn’t think he was taking me seriously. “I hear you, and I truly apologize if my previous response came across as anything less than sincere,” Reece said. “I absolutely recognize that Escape From Tarkov is far more than just a game to its community.”

“Some poor kid in [Kentucky] won't be able to brush their teeth tonight because of the commitment to the joke I had,” Reece said, letting go of the bit and joking about the massive amounts of water AI datacenters use.

Getting made fun of by my real friends, even when they’re using LLMs to do it, was way better than any snide remark Wolf made. I’d rather play solo, for all its struggles and loneliness, than stare anymore into that AI-generated abyss.

Even the U.S. Government Says AI Requires Massive Amounts of Water

A new government illuminates the environmental impact of generative AI.

^{Matthew Gault (404 Media)}

404 Media

2026-01-20 17:47:24

How Wikipedia Will Survive in the Age of AI (With Wikipedia’s CTO Selena Deckelmann)

Wikipedia is turning 25 this month, and it’s never been more important.

The online, collectively created encyclopedia has been a cornerstone of the internet decades, but as generative AI started flooding every platform with AI-generated slop over the last couple of years, Wikipedia’s governance model, editing process, and dedication to citing reliable sources has emerged as one of the most reliable and resilient models we have.

And yet, as successful as the model is, it’s almost never replicated.
open.spotify.com/embed/episode…
This week on the podcast we’re joined by Selena Deckelmann, the Chief Product and Technology Officer at the Wikimedia Foundation, the nonprofit organization that operates Wikipedia. That means Selena oversees the technical infrastructure and product strategy for one of the most visited sites in the world, and one the most comprehensive repositories of human knowledge ever assembled. Wikipedia is turning 25 this month, so I wanted to talk to Selena about how Wikipedia works and how it plans to continue to work in the age of generative AI.

Listen to the weekly podcast on Apple Podcasts, Spotify, or YouTube.

Become a paid subscriber for early access to these interview episodes and to power our journalism. If you become a paid subscriber, check your inbox for an email from our podcast host Transistor for a link to the subscribers-only version! You can also add that subscribers feed to your podcast app of choice and never miss an episode that way. The email should also contain the subscribers-only unlisted YouTube link for the extended video version too. It will also be in the show notes in your podcast player.

• Wikipedia’s value in the age of generative AI
• The Editors Protecting Wikipedia from AI Hoaxes
• Wikipedia Pauses AI-Generated Summaries After Editor Backlash
• Wikipedia Says AI Is Causing a Dangerous Decline in Human Visitors
• Jimmy Wales Says Wikipedia Could Use AI. Editors Call It the 'Antithesis of Wikipedia'

youtube.com/embed/39LR9ouJR3c?…

Wikipedia Says AI Is Causing a Dangerous Decline in Human Visitors

“With fewer visits to Wikipedia, fewer volunteers may grow and enrich the content, and fewer individual donors may support this work.”

^{Emanuel Maiberg (404 Media)}

404 Media

2025-12-18 17:19:25

ICE Contracts Company Making Bounty Hunter AI Agents

Immigration and Customs Enforcement (ICE) has paid hundreds of thousands of dollars to a company that makes “AI agents” to rapidly track down targets. The company claims the “skip tracing” AI agents help agencies find people of interest and map out their family and other associates more quickly. According to the procurement records, the company’s services were specifically for Enforcement and Removal Operations (ERO), the part of ICE that identifies, arrests, and deports people.

The contract comes as ICE is spending millions of dollars, and plans to spend tens of millions more, on skip tracing services more broadly. The practice involves ICE paying bounty hunters to use digital tools and physically stalk immigrants to verify their addresses, then report that information to ICE so the agency can act.

This post is for subscribers only

Become a member to get access to all content
Subscribe now

ICE Contracts Company Making Bounty Hunter AI Agents

^{Joseph Cox (404 Media)}

404 Media

2025-12-11 15:56:21

Porn Is Being Injected Into Government Websites Via Malicious PDFs

Dozens of government and university websites belonging to cities, towns, and public agencies across the country are hosting PDFs promoting AI porn apps, porn sites, and cryptocurrency scams; dozens more have been hit with a website redirection attacks which lead to animal vagina sex toy ecommerce pages, penis enlargement treatments, automatically-downloading Windows program files, and porn.

“Sex xxx video sexy Xvideo bf porn XXX xnxx Sex XXX porn XXX blue film Sex Video xxx sex videos Porn Hub XVideos XXX sexy bf videos blue film Videos Oficial on Instagram New Viral Video The latest original video has taken the internet by storm and left viewers in on various social media platforms ex Videos Hot Sex Video Hot Porn viral video,” reads the beginning of a three-page PDF uploaded to the website of the Irvington, New Jersey city government’s website.

The PDF, called “XnXX Video teachers fucking students Video porn Videos free XXX Hamster XnXX com” is unlike many of the other PDFs hosted on the city’s website, which include things like “2025-10-14 Council Minutes,” “Proposed Agenda 9-22-25,” and “Landlord Registration Form (1 & 2 unit dwelling).”

It is similar, however, to another PDF called “30 Best question here’s,” which looks like this:

Irvington, which is just west of Newark and has a population of 61,000 people, has fallen victim to an SEO spam attack that has afflicted local and state governments and universities around the United States.

💡
Do you know anything else about whatever is going on here? I would love to hear from you. Using a non-work device, you can message me securely on Signal at jason.404. Otherwise, send me an email at jason@404media.co.

Researcher Brian Penny has identified dozens of government and university websites that hosted PDF guides for how to make AI porn, PDFs linking to porn videos, bizarre crypto spam, sex toys, and more.

Reginfo.gov, a regulatory affairs compliance website under the federal government’s General Services Administration, is currently hosting a 12 page PDF called “Nudify AI Free, No Sign-Up Needed!,” which is an ad and link to an abusive AI app designed to remove a person’s clothes. The Kansas Attorney General’s office and the Mojave Desert Air Quality Management District Office in California hosted PDFs called “DeepNude AI Best Deepnude AI APP 2025.” Penny found similar PDFs on the websites for the Washington Department of Fish and Wildlife, the Washington Fire Commissioners Association, the Florida Department of Agriculture, the cities of Jackson, Mississippi and Massillon, Ohio, various universities throughout the country, and dozens of others. Penny has caught the attention of local news throughout the United States, who have reported on the problem.

The issue appears to be stemming from websites that allow people to upload their own PDFs, which then sit on these government websites. Because they are loaded with keywords for widely searched terms and exist on government and university sites with high search authority, Google and other search engines begin to surface them. In the last week or so, many (but not all) of the PDFs Penny has discovered have been deleted by local governments and universities.

But cities seem like they are having more trouble cleaning up another attack, which is redirecting traffic from government URLs to porn, e-commerce, and spam sites. In an attack that seems similar to what we reported in June, various government websites are somehow being used to maliciously send traffic elsewhere. For example, the New York State Museum’s online exhibit for something called “The Family Room” now has at least 11 links to different types of “realistic” animal vagina pocket masturbators, which include “Zebra Animal Vagina Pussy Male Masturbation Cup — Pocket Realistic Silicone Penis Sex Toy ($27.99),” and “Must-have Horse Pussy Torso Buttocks Male Masturbator — Fantasy Realistic Animal Pussie Sex Doll.”

Links Penny found on Knoxville, Tennessee’s site for permitting inspections first go to a page that looks like a government site for hosting files then redirects to a page selling penis growth supplements that features erect penises (human penises, mercifully), blowjobs, men masturbating, and Dr. Oz’s face.

Another Knoxville link I found, which purports to be a pirated version of the 2002 Vin Diesel film XXX simply downloaded a .exe file to my computer.

Penny believes that what he has found is basically the tip of the iceberg, because he is largely finding these by typing things like “nudify site:.gov” “xxx site:.gov” into Google and clicking around. Sometimes, malicious pages surface only on image searches or video searches: “Basically the craziest things you can think of will show up as long as you’re on image search,” Penny told 404 Media. “I’ll be doing this all week.”

The Nevada Department of Transportation told 404 Media that “This incident was not related to NDOT infrastructure or information systems, and the material was not hosted on NDOT servers.This unfortunate incident was a result of malicious use of a legitimate form created using the third-party platform on which NDOT’s website is hosted. NDOT expeditiously worked with our web hosting vendor to ensure the inappropriate content was removed.” It added that the third-party is Granicus, a massive government services company that provides website backend infrastructure for many cities and states around the country, as well as helps them stream and archive city council meetings, among other services. Several of the affected local governments use Granicus, but not all of them do; Granicus did not respond to two requests for comment from 404 Media.

The California Secretary of State’s Office told 404 Media: “A bad actor uploaded non-business documents to the bizfile Online system (a portal for business filings and information). The files were then used in external links allowing public access to only those uploaded files. No data was compromised. SOS staff took immediate action to remove the ability to use the system for non-SOS business purposes and are removing the unauthorized files from the system.” The Washington Department of Fish and Wildlife said “WDFW is aware of this issue and is actively working with our partners at WaTech to address it.” The other government agencies mentioned in this article did not respond to our requests for comment.

AI porn on state websites

Mainland whistleblower warns of malicious webpages being hosted on government websites nationwide, including in Hawaiʻi.

^{Michael Brestovansky (Aloha State Daily)}