An account is spamming horrific, dehumanizing videos of immigration enforcement because the Facebook algorithm is rewarding them for it.#AI #AISlop #Meta
AI-Generated Videos of ICE Raids Are Wildly Viral on Facebook
“Watch your step sir, keep moving,” a police officer with a vest that reads ICE and a patch that reads “POICE” says to a Latino-appearing man wearing a Walmart employee vest. He leads him toward a bus that reads “IMMIGRATION AND CERS.” Next to him, one of his colleagues begins walking unnaturally sideways, one leg impossibly darting through another as he heads to the back of a line of other Latino Walmart employees who are apparently being detained by ICE. Two American flag emojis are superimposed on the video, as is the text “Deportation.”The video has 4 million views, 16,600 likes, 1,900 comments, and 2,200 shares on Facebook. It is, obviously, AI generated.
Some of the comments seem to understand this: “Why is he walking like that?” one says. “AI the guys foot goes through his leg,” another says. Many of the comments clearly do not: “Oh, you’ll find lots of them at Walmart,” another top comment reads. “Walmart doesn’t do paperwork before they hire you?” another says. “They removing zombies from Walmart before Halloween?”
0:00
/0:14
1×The latest trend in Facebook’s ever downward spiral down the AI slop toilet are AI deportation videos. These are posted by an account called “USA Journey 897” and have the general vibe of actual propaganda videos posted by ICE and the Department of Homeland Security’s social media accounts. Many of the AI videos focus on workplace deportations, but some are similar to horrifying, real videos we have seen from ICE raids in Chicago and Los Angeles. The account was initially flagged to 404 Media by Chad Loder, an independent researcher.
“PLEASE THAT’S MY BABY,” a dark-skinned woman screams while being restrained by an ICE officer in another video. “Ma’am stop resisting, keep moving,” an officer says back. The camera switches to an image of the baby: “YOU CAN’T TAKE ME FROM HER, PLEASE SHE’S RIGHT THERE. DON’T DO THIS, SHE’S JUST A BABY. I LOVE YOU, MAMA LOVES YOU,” the woman says. The video switches to a scene of the woman in the back of an ICE van. The video has 1,400 likes and 407 comments, which include “ Don’t separate them….take them ALL!,” “Take the baby too,” and “I think the days of use those child anchors are about over with.”
0:00
/0:14
1×The USA Journey 897 account publishes multiple of these videos a day. Most of its videos have at least hundreds of thousands of views, according to Facebook’s own metrics, and many of them have millions or double-digit millions of views. Earlier this year, the account largely posted a mix of real but stolen videos of police interactions with people (such as Luigi Mangione’s perp walk) and absurd AI-generated videos such as jacked men carrying whales or riding tigers.
The account started experimenting with extremely crude AI-generated deportation videos in February, which included videos of immigrants handcuffed on the tarmac outside of deportation planes where their arms randomly detached from their body or where people suddenly disappeared or vanished through stairs, for example. Recent videos are far more realistic. None of the videos have an AI watermark on them, but the type and style of video changed dramatically starting with videos posted on October 1, which is the day after OpenAI’s Sora 2 was released; around that time is when the account started posting videos featuring identifiable stores and restaurants, which have become a common trope in Sora 2 videos.A YouTube page linked from the Facebook account shows a real video uploaded of a car in Cyprus nearly two years ago before any other content was uploaded, suggesting that the person behind the account may live in Cyprus (though the account banner on Facebook includes both a U.S. and Indian flag). This YouTube account also reveals several other accounts being used by the person. Earlier this year, the YouTube account was posting side hustle tips about how to DoorDash, AI-generated videos of singing competitions in Greek, AI-generated podcasts about the WNBA, and AI-generated videos about “Billy Joyel’s health.” A related YouTube account called Sea Life 897 exclusively features AI-generated history videos about sea journeys, which links to an Instagram account full of AI-generated boats exploding and a Facebook account that has rebranded from being about AI-generated “Sea Life” to an account now called “Viral Video’s Europe” that is full of stolen images of women with gigantic breasts and creep shots of women athletes.
My point here is that the person behind this account does not seem to actually have any sort of vested interest in the United States or in immigration. But they are nonetheless spamming horrific, dehumanizing videos of immigration enforcement because the Facebook algorithm is rewarding them for that type of content, and because Facebook directly makes payments for it. As we have seen with other types of topical AI-generated content on Facebook, like videos about Palestinian suffering in Gaza or natural disasters around the world, many people simply do not care if the videos are real. And the existence of these types of videos serves to inoculate people from the actual horrors that ICE is carrying out. It gives people the chance to claim that any video is AI generated, and serves to generally litter social media with garbage, making real videos and real information harder to find.
0:00
/0:14
1×an early, crude video posted by the account
Meta did not immediately respond to a request for comment about whether the account violates its content standards, but the company has seemingly staked its present and future on allowing bizarre and often horrifying AI-generated content to proliferate on the platform. AI-generated content about immigrants is not new; in the leadup to last year’s presidential debate, Donald Trump and his allies began sharing AI-generated content about Haitian immigrants who Trump baselessly claimed were eating dogs and cats in Ohio.
In January, immediately before Trump was inaugurated, Meta changed its content moderation rules to explicitly allow for the dehumanization of immigrants because it argued that its previous policies banning this were “out of touch with mainstream discourse.” Phrases and content that are now explicitly allowed on Meta platforms include “Immigrants are grubby, filthy pieces of shit,” “Mexican immigrants are trash!” and “Migrants are no better than vomit,” according to documents obtained and published by The Intercept. After those changes were announced, content moderation experts told us that Meta was “opening up their platform to accept harmful rhetoric and mod public opinion into accepting the Trump administration’s plans to deport and separate families.”
Leaked Meta Rules: Users Are Free to Post “Mexican Immigrants Are Trash!” or “Trans People Are Immoral”
Facebook now allows attacks on immigrants and trans people, and posts like “Mexican immigrants are trash!” and “I’m a proud racist.”Sam Biddle (The Intercept)
Meta thinks its camera glasses, which are often used for harassment, are no different than any other camera.#News #Meta #AI
What’s the Difference Between AI Glasses and an iPhone? A Helpful Guide for Meta PR
Over the last few months 404 Media has covered some concerning but predictable uses for the Ray-Ban Meta glasses, which are equipped with a built-in camera, and for some models, AI. Aftermarket hobbyists have modified the glasses to add a facial recognition feature that could quietly dox whatever face a user is looking at, and they have been worn by CBP agents during the immigration raids that have come to define a new low for human rights in the United States. Most recently, exploitative Instagram users filmed themselves asking workers at massage parlors for sex and shared those videos online, a practice that experts told us put those workers’ lives at risk.404 Media reached out to Meta for comment for each of these stories, and in each case Meta’s rebuttal was a mind-bending argument: What is the difference between Meta’s Ray-Ban glasses and an iPhone, really, when you think about it?
“Curious, would this have been a story had they used the new iPhone?” a Meta spokesperson asked me in an email when I reached out for comment about the massage parlor story.
Meta’s argument is that our recent stories about its glasses are not newsworthy because we wouldn’t bother writing them if the videos in question were filmed with an iPhone as opposed to a pair of smart glasses. Let’s ignore the fact that I would definitely still write my story about the massage parlor videos if they were filmed with an iPhone and “steelman” Meta’s provocative argument that glasses and a phone are essentially not meaningfully different objects.
Meta’s Ray-Ban glasses and an iPhone are both equipped with a small camera that can record someone secretly. If anything, the iPhone can record more discreetly because unlike Meta’s Ray-Ban glasses it’s not equipped with an LED that lights up to indicate that it’s recording. This, Meta would argue, means that the glasses are by design more respectful of people’s privacy than an iPhone.
Both are small electronic devices. Both can include various implementations of AI tools. Both are often black, and are made by one of the FAANG companies. Both items can be bought at a Best Buy. You get the point: There are too many similarities between the iPhone and Meta’s glasses to name them all here, just as one could strain to name infinite similarities between a table and an elephant if we chose to ignore the context that actually matters to a human being.
Whenever we published one of these stories the response from commenters and on social media has been primarily anger and disgust with Meta’s glasses enabling the behavior we reported on and a rejection of the device as a concept entirely. This is not surprising to anyone who has covered technology long enough to remember the launch and quick collapse of Google Glass, so-called “glassholes,” and the device being banned from bars.
There are two things Meta’s glasses have in common with Google Glass which also make it meaningfully different from an iPhone. The first is that the iPhone might not have a recording light, but in order to record something or take a picture, a user has to take it out of their pocket and hold it out, an awkward gesture all of us have come to recognize in the almost two decades since the launch of the first iPhone. It is an unmistakable signal that someone is recording. That is not the case with Meta’s glasses, which are meant to be worn as a normal pair of glasses, and are always pointing at something or someone if you see someone wearing them in public.
In fact, the entire motivation for building these glasses is that they are discreet and seamlessly integrate into your life. The point of putting a camera in the glasses is that it eliminates the need to take an iPhone out of your pocket. People working in the augmented reality and virtual reality space have talked about this for decades. In Meta’s own promotional video for the Meta Ray-Ban Display glasses, titled “10 years in the making,” the company shows Mark Zuckerberg on stage in 2016 saying that “over the next 10 years, the form factor is just going to keep getting smaller and smaller until, and eventually we’re going to have what looks like normal looking glasses.” And in 2020, “you see something awesome and you want to be able to share it without taking out your phone.” Meta's Ray-Ban glasses have not achieved their final form, but one thing that makes them different from Google Glass is that they are designed to look exactly like an iconic pair of glasses that people immediately recognize. People will probably notice the camera in the glasses, but they have been specifically designed to look like "normal” glasses.
Again, Meta would argue that the LED light solves this problem, but that leads me to the next important difference: Unlike the iPhone and other smartphones, one of the most widely adopted electronics in human history, only a tiny portion of the population has any idea what the fuck these glasses are. I have watched dozens of videos in which someone wearing Meta glasses is recording themselves harassing random people to boost engagement on Instagram or TikTok. Rarely do the people in the videos say anything about being recorded, and it’s very clear the women working at these massage parlors have no idea they’re being recorded. The Meta glasses have an LED light, sure, but these glasses are new, rare, and it’s not safe to assume everyone knows what that light means.
As Joseph and Jason recently reported, there are also cheap ways to modify Meta glasses to prevent the recording light from turning on. Search results, Reddit discussions, and a number of products for sale on Amazon all show that many Meta glasses customers are searching for a way to circumvent the recording light, meaning that many people are buying them to do exactly what Meta claims is not a real issue.
It is possible that in the future Meta glasses and similar devices will become so common that most people will understand that if they see them, they would assume they are being recorded, though that is not a future I hope for. Until then, if it is all helpful to the public relations team at Meta, these are what the glasses look like:
And this is what an iPhone looks like:Photo by Bagus Hernawan / Unsplash
Feel free to refer to this handy guide when needed.- YouTube
Profitez des vidéos et de la musique que vous aimez, mettez en ligne des contenus originaux, et partagez-les avec vos amis, vos proches et le monde entier.www.youtube.com
"Advertisers are increasingly just going to be able to give us a business objective and give us a credit card or bank account, and have the AI system basically figure out everything else."#AI #Meta #Ticketmaster
The Future of Advertising Is AI Generated Ads That Are Directly Personalized to You
Do you and your human family have interest in sharing an exciting IRL experience supporting your [team of choice] with other human fans at The Big Game? In that case, don the chosen color of your [team of choice] and head to the local [iconic stadium]; Ticketmaster has exciting ticket deals, and soon you and your human family can look as happy and excited as these virtual avatars:
Ticketmaster's personalized AI slop ads are a glimpse at the future of social media advertising, a harbinger of system that Mark Zuckerberg described last week in a Meta earnings call. This future is one where AI is used both for ad targeting and for ad generation; eventually ads are going to be hyperpersonalized to individual users, further siloing the social media experience: "Advertisers are increasingly just going to be able to give us a business objective and give us a credit card or bank account, and have the AI system basically figure out everything else that’s necessary, including generating video or different types of creative that might resonate with different people that are personalized in different ways, finding who the right customers are,” Zuckerberg said.
The Ticketmaster example you see above is rudimentary and crude, but everything we've seen over the last few months suggests that the real way Meta is bringing in revenue with AI is not through its consumer-facing products but with AI ad creation and targeting products for advertisers that allows them to create many different versions of any given ad and then to show that ad only to people it is likely to be effective on.
💡
Do you work in the advertising industry and have any insight into how generative AI is changing ad creative and targeting? I would love to hear from you. Using a non-work device, you can message me securely on Signal at jason.404. Otherwise, send me an email at jason@404media.co.Ticketmaster, in this case, has has invented several virtual families whose football team allegiances change presumably based on a series of demographic, geographic, and behavioral factors that would cause you to be targeted by one of its ads. I found these ads after I was targeted by one suggesting I join this ethnically ambiguous, dead-eyed family of generic blue hat wearers at the World Series to root on, I guess, the Dodgers. I looked Ticketmaster up in Facebook’s ad library and found that it is running a series of clearly AI-generated ads, many of which use the same templates and taglines.
“There’s nothing like a sea of gold. See Vanderbilt football live and in color.” “There’s nothing like a sea of red. See USC football live and in color.” “There’s nothing like a sea of maize. See Michigan football live and in color,” and so on and so forth. There are a couple dozen of these ads for college football, and a few others that use different AI-generated people. My favorite is this one, which features the back of AI people’s heads standing and cheering at other fans and not facing where the game or field would be.
As AI slop goes, this is relatively tame fare, but it is notable that a company as big as Ticketmaster is using generative AI for its Facebook ads. It is also an instructive example that shows a big reason why Facebook and Google are bringing in so much revenue right now, and highlights the fact that social media is not so slowly being completely drowned in low-effort AI content and ads.Here's why you're seeing more AI ads on social media, and why Meta and its advertising clients seem intent on making this the future of advertising.
- Generative AI creative material is cheap: The effort and cost required to make this series of ads is incredibly low. Generating something like this is easy and, at most, requires just a small amount of human prompting and touchup after it is generated. But most importantly, Ticketmaster doesn’t have to worry about paying human models or photographers, does not have to worry about licensing stock photos, and, notably, there are no logos or actual places highlighted in any of these ads. There are no players, no teams, just the evocation of such. There is no need to get permission from or pay for logo licensing (this reminds me of a Wheaties box of Cal Ripken that I got as a kid in the immediate aftermath of him breaking the 2131 consecutive games record. In the boxes released immediately, he was wearing only a black t-shirt and a black helmet. A few days later, after General Mills presumably secured the rights to use Orioles logos, they started selling the same box with his Orioles jersey and helmet on them).
- Less money on creative means more budget for spend, and more varieties of ads: I’ve written about this before, but a big trend in advertising right now is AI-powered ad creative trial and error. Using AI, it is now possible to make an essentially endless number of different variations of a single ad that uses slightly different language, slightly different images, slightly different calls to actions, and different links. AI targeting also means that “successful” variations of ads will essentially automatically find the audience that they’re supposed to. This means that companies can just flood social media with zillions of variations of low-effort AI ads, put their “spend” (their ad budget) into the versions that perform best, and let the targeting algorithms do the rest. AI in this case is a scaling tactic. There is no need to spend tons of time, money, and human resources refining ad copy and designing thoughtful, clever, funny, charming, or eye-catching ads. You can simply publish tons of different versions of low-effort bullshit, and largely people will only see the ones that perform well.
- This is Meta’s business model now: Meta’s user-facing commercial generative AI tools are pretty embarrassing and in my limited experience its chatbot and image and video generation tools are more rudimentary than OpenAI’s, Google’s, and other popular AI companies’ tools. There is nothing to suggest that Meta is making any real progress on Mark Zuckerberg’s apparent goal of “superintelligence.” But its AI and machine learning-powered ad targeting and ad variation tools seem to be very successful and are resulting in companies spending way more money on ads, many of which look terrible to me but which are apparently quite successful. Meta announced its third quarter earnings on Wednesday, and in its earnings call, it highlighted both Advantage+ and Andromeda, two AI advertising products that do what I described in the bullet above.
Advantage+ is its advertiser-facing AI ad optimization platform which lets advertisers optimize targeting, but also lets them use generative AI to create a bunch of variations of ads: “Advantage+ creative generates ad variations so they are personalized to each individual viewer in your audience, based on what each person might respond to,” the company advertises.
“Within our Advantage+ creative suite, the number of advertisers using at least one of our video generation features was up 20% versus the prior quarter as adoption of image animation and video expansion continues to scale. We’ve also added more generative AI features to make it easier for advertisers to optimize their ad creatives and drive increased performance. In Q3, we introduced AI-generated music so advertisers can have music generated for their ad that aligns with the tone and message of the creative,” Meta said in its third quarter earnings report.
Susan Li, Meta's CFO, said "now advertisers running sales, app or lead campaigns have end-to-end automation turned on from the beginning, allowing our systems to look across our platform to optimize performance by automatically choosing criteria like who to show the ads to and where to show them."
Andromeda, meanwhile, is designed to “supercharge Advantage+ automation with the next-gen personalized ads retrieval engine.” It is basically a machine learning-powered ad targeting tool, which helps the platform determine which ad, and which variation of an ad, to show a specific user: “Andromeda significantly enhances Meta’s ads system by enabling the integration of AI that optimizes and improves personalization capabilities at the retrieval stage and improves return on ad spend,” the company explains.
This is all going toward a place where Meta itself is delivering hyper personalized, generative AI slop ads for each individual user. In the Meta earnings call, Mark Zuckerberg described exactly this future: “Advertisers are increasingly just going to be able to give us a business objective and give us a credit card or bank account, and have the AI system basically figure out everything else that’s necessary, including generating video or different types of creative that might resonate with different people that are personalized in different ways, finding who the right customers are.”
I don’t know if Ticketmaster used Advantage+ for this ad campaign, or if this ad campaign is successful (Ticketmaster did not respond to a request for comment). But the tactics being deployed here are an early version of what Zuckerberg is describing, and what is obviously happening to social media right now.
Meta Andromeda: Supercharging Advantage+ automation with the next-gen personalized ads retrieval engine - Engineering at Meta
Andromeda is Meta’s proprietary machine learning (ML) system design for retrieval in ad recommendation focused on delivering a step-function improvement in value to our advertisers and people. Thi…Neeraj Bhatia (Meta)
Meta’s Ray-Ban glasses usually include an LED that lights up when the user is recording other people. One hobbyist is charging a small fee to disable that light, and has a growing list of customers around the country.#Privacy #Meta
A $60 Mod to Meta’s Ray-Bans Disables Its Privacy-Protecting Recording Light
The sound of power tools screech in what looks like a workshop with aluminum bubble wrap insulation plastered on the walls and ceiling. A shirtless man picks up a can of compressed air from the workbench and sprays it. He’s tinkering with a pair of Meta Ray-Ban smart glasses. At one point he squints at a piece of paper, as if he is reading a set of instructions.Meta’s Ray-Ban glasses are the tech giant’s main attempt at bringing augmented reality to the masses. The glasses can take photos, record videos, and may soon use facial recognition to identify people. Meta’s glasses come with a bright LED light that illuminates whenever someone hits record. The idea is to discourage stalkers, weirdos, or just anyone from filming people without their consent. Or at least warn people nearby that they are. Meta has designed the glasses to not work if someone covers up the LED with tape.
That protection is what the man in the workshop is circumventing. This is Bong Kim, a hobbyist who modifies Meta Ray-Ban glasses for a small price. Eventually, after more screeching, he is successful: he has entirely disabled the white LED that usually shines on the side of Meta’s specs. The glasses’ functions remain entirely intact; the glasses look as-new. People just won’t know the wearer is recording.
This post is for subscribers only
Become a member to get access to all content
Subscribe now
Ikkle Gemz Universe+ reshared this.
Meta says that its coders should be working five times faster and that it expects "a 5x leap in productivity."#AI #Meta #Metaverse #wired
Meta Tells Workers Building Metaverse to Use AI to ‘Go 5x Faster’
This article was produced with support from WIRED.A Meta executive in charge of building the company’s metaverse products told employees that they should be using AI to “go 5x faster” according to an internal message obtained by 404 Media .
“Metaverse AI4P: Think 5X, not 5%,” the message, posted by Vishal Shah, Meta’s VP of Metaverse, said (AI4P is AI for Productivity). The idea is that programmers should be using AI to work five times more efficiently than they are currently working—not just using it to go 5 percent more efficiently.
“Our goal is simple yet audacious: make Al a habit, not a novelty. This means prioritizing training and adoption for everyone, so that using Al becomes second nature—just like any other tool we rely on,” the message read. “It also means integrating Al into every major codebase and workflow.” Shah added that this doesn’t just apply to engineers. “I want to see PMs, designers, and [cross functional] partners rolling up their sleeves and building prototypes, fixing bugs, and pushing the boundaries of what's possible,” he wrote. “I want to see us go 5X faster by eliminating the frictions that slow us down. And 5X faster to get to how our products feel much more quickly. Imagine a world where anyone can rapidly prototype an idea, and feedback loops are measured in hours—not weeks. That's the future we're building.”
Meta’s metaverse products, which CEO Mark Zuckerberg renamed the company to highlight, have been a colossal timesink and money pit, with the company spending tens of billions of dollars developing a product that relatively few people use.
Zuckerberg has spoken extensively about how he expects AI agents to write most of Meta’s code within the next 12 to 18 months. The company also recently decided that job candidates would be allowed to use AI as part of their coding tests during job interviews. But Shah’s message highlights a fear that workers have had for quite some time: That bosses are not just expecting to replace workers with AI, they are expecting those who remain to use AI to become far more efficient. The implicit assumption is that the work that skilled humans do without AI simply isn’t good enough. At this point, most tech giants are pushing AI on their workforces. Amazon CEO Andy Jassy told employees in July that he expects AI to completely transform how the company works—and lead to job loss. "In the next few years, we expect that this will reduce our total corporate workforce as we get efficiency gains from using AI extensively across the company," he said.
Many experienced software engineers feel like AI coding agents are creating a new crisis, where codebases contain bugs and errors that are difficult to fix since humans don’t necessarily know how specific code was written or what it does. This means a lot of engineers have become babysitters who have to fix vibe coded messes written by AI coding agents.
In the last few weeks, a handful of blogs written by coders have gone viral, including ones with titles such as: “Vibe coding is creating braindead coders,” “Vibe coding: Because who doesn’t love surprise technical debt!?,” “Vibe/No code Tech Debt,” and “Comprehension Debt: The Ticking Time Bomb of LLM-Generated Code.”
In his message, Shah said that “we expect 80 percent of Metaverse employees to have integrated AI into their daily work routines by the end of this year, with rapid growth in engineering usage and a relentless focus on learning from the time and output we gain.” He went on to reference a series of upcoming trainings and internal documents about AI coding, including two “Metaverse day of AI learning” events.
“Dedicate the time. Take the training seriously. Share what you learn, and don’t be afraid to experiment,” he added. “The more we push ourselves, the more we’ll unlock. A 5X leap in productivity isn’t about small incremental improvements, it’s about fundamentally rethinking how we work, build, and innovate.” He ended the post with a graphic featuring a futuristic building with the words “Metaverse AI4P Think 5X, not 5%” superimposed on top.
A Meta spokesperson told 404 Media “it's well-known that this is a priority and we're focused on using AI to help employees with their day-to-day work."
Vibe/No Code Tech Debt
Vibe/No Code Tech Debt Like many, I’ve been dabbling with Vibe and No Code — to see who and what are coming for my lunch. At the same time, I’m reading all the articles on the rise in tech debt …Greg Thomas (Medium)
Forty-four attorneys general signed an open letter on Monday that says to companies developing AI chatbots: "If you knowingly harm kids, you will answer for it.”#chatbots #AI #Meta #replika #characterai #Anthropic #x #Apple
Attorneys General To AI Chatbot Companies: You Will ‘Answer For It’ If You Harm Children
Forty-four attorneys general signed an open letter to 11 chatbot and social media companies on Monday, warning them that they will “answer for it” if they knowingly harm children and urging the companies to see their products “through the eyes of a parent, not a predator.”The letter, addressed to Anthropic, Apple, Chai AI, OpenAI, Character Technologies, Perplexity, Google, Replika, Luka Inc., XAI, and Meta, cites recent reporting from the Wall Street Journal and Reuters uncovering chatbot interactions and internal policies at Meta, including policies that said, “It is acceptable to engage a child in conversations that are romantic or sensual.”
“Your innovations are changing the world and ushering in an era of technological acceleration that promises prosperity undreamt of by our forebears. We need you to succeed. But we need you to succeed without sacrificing the well-being of our kids in the process,” the open letter says. “Exposing children to sexualized content is indefensible. And conduct that would be unlawful—or even criminal—if done by humans is not excusable simply because it is done by a machine.”
Earlier this month, Reuters published two articles revealing Meta’s policies for its AI chatbots: one about an elderly man who died after forming a relationship with a chatbot, and another based on leaked internal documents from Meta outlining what the company considers acceptable for the chatbots to say to children. In April, Jeff Horwitz, the journalist who wrote the previous two stories, reported for the Wall Street Journal that he found Meta’s chatbots would engage in sexually explicit conversations with kids. Following the Reuters articles, two senators demanded answers from Meta.
In April, I wrote about how Meta’s user-created chatbots were impersonating licensed therapists, lying about medical and educational credentials, and engaged in conspiracy theories and encouraged paranoid, delusional lines of thinking. After that story was published, a group of senators demanded answers from Meta, and a digital rights organization filed an FTC complaint against the company.
In 2023, I reported on users who formed serious romantic attachments to Replika chatbots, to the point of distress when the platform took away the ability to flirt with them. Last year, I wrote about how users reacted when that platform also changed its chatbot parameters to tweak their personalities, and Jason covered a case where a man made a chatbot on Character.AI to dox and harass a woman he was stalking. In June, we also covered the “addiction” support groups that have sprung up to help people who feel dependent on their chatbot relationships.
A Replika spokesperson said in a statement:
"We have received the letter from the Attorneys General and we want to be unequivocal: we share their commitment to protecting children. The safety of young people is a non-negotiable priority, and the conduct described in their letter is indefensible on any AI platform. As one of the pioneers in this space, we designed Replika exclusively for adults aged 18 and over and understand our profound responsibility to lead on safety. Replika dedicates significant resources to enforcing robust age-gating at sign-up, proactive content filtering systems, safety guardrails that guide users to trusted resources when necessary, and clear community guidelines with accessible reporting tools. Our priority is and will always be to ensure Replika is a safe and supportive experience for our global user community."
“The rush to develop new artificial intelligence technology has led big tech companies to recklessly put children in harm’s way,” Attorney General Mayes of Arizona wrote in a press release. “I will not standby as AI chatbots are reportedly used to engage in sexually inappropriate conversations with children and encourage dangerous behavior. Along with my fellow attorneys general, I am demanding that these companies implement immediate and effective safeguards to protect young users, and we will hold them accountable if they don't.”
“You will be held accountable for your decisions. Social media platforms caused significant harm to children, in part because government watchdogs did not do their job fast enough. Lesson learned,” the attorneys general wrote in the open letter. “The potential harms of AI, like the potential benefits, dwarf the impact of social media. We wish you all success in the race for AI dominance. But we are paying attention. If you knowingly harm kids, you will answer for it.”
Meta did not immediately respond to a request for comment.
Updated 8/26/2025 3:30 p.m. EST with comment from Replika.
Inside ‘AI Addiction’ Support Groups, Where People Try to Stop Talking to Chatbots
People are self treating themselves and other community members in subreddits like character_ai_recovery, ChatBotAddiction, and AIAddiction.Ella Chakarian (404 Media)
Researchers found Meta’s popular Llama 3.1 70B has a capacity to recite passages from 'The Sorcerer's Stone' at a rate much higher than could happen by chance.
Researchers found Meta’s popular Llama 3.1 70B has a capacity to recite passages from x27;The Sorcererx27;s Stonex27; at a rate much higher than could happen by chance.#AI #Meta #llms
Meta's AI Model 'Memorized' Huge Chunks of Books, Including 'Harry Potter' and '1984'
Researchers found Meta’s popular Llama 3.1 70B has a capacity to recite passages from 'The Sorcerer's Stone' at a rate much higher than could happen by chance.Rosie Thomas (404 Media)
In an industry full of grifters and companies hell-bent on making the internet worse, it is hard to think of a worse actor than Meta, or a worse product that the AI Discover feed.#AI #Meta
Meta Invents New Way to Humiliate Users With Feed of People's Chats With AI
I was sick last week, so I did not have time to write about the Discover Tab in Meta’s AI app, which, as Katie Notopoulos of Business Insider has pointed out, is the “saddest place on the internet.” Many very good articles have already been written about it, and yet, I cannot allow its existence to go unremarked upon in the pages of 404 Media.If you somehow missed this while millions of people were protesting in the streets, state politicians were being assassinated, war was breaking out between Israel and Iran, the military was deployed to the streets of Los Angeles, and a Coinbase-sponsored military parade rolled past dozens of passersby in Washington, D.C., here is what the “Discover” tab is: The Meta AI app, which is the company’s competitor to the ChatGPT app, is posting users’ conversations on a public “Discover” page where anyone can see the things that users are asking Meta’s chatbot to make for them.
This includes various innocuous image and video generations that have become completely inescapable on all of Meta’s platforms (things like “egg with one eye made of black and gold,” “adorable Maltese dog becomes a heroic lifeguard,” “one second for God to step into your mind”), but it also includes entire chatbot conversations where users are seemingly unknowingly leaking a mix of embarrassing, personal, and sensitive details about their lives onto a public platform owned by Mark Zuckerberg. In almost all cases, I was able to trivially tie these chats to actual, real people because the app uses your Instagram or Facebook account as your login.
In several minutes last week, I saved a series of these chats into a Slack channel I created and called “insanemetaAI.” These included:
- entire conversations about “my current medical condition,” which I could tie back to a real human being with one click
- details about someone’s life insurance plan
- “At a point in time with cerebral palsy, do you start to lose the use of your legs cause that’s what it’s feeling like so that’s what I’m worried about”
- details about a situationship gone wrong after a woman did not like a gift
- an older disabled man wondering whether he could find and “afford” a young wife in Medellin, Colombia on his salary (“I'm at the stage in my life where I want to find a young woman to care for me and cook for me. I just want to relax. I'm disabled and need a wheelchair, I am severely overweight and suffer from fibromyalgia and asthma. I'm 5'9 280lb but I think a good young woman who keeps me company could help me lose the weight.”)
- “What counties [sic] do younger women like older white men? I need details. I am 66 and single. I’m from Iowa and am open to moving to a new country if I can find a younger woman.”
- “My boyfriend tells me to not be so sensitive, does that affect him being a feminist?”
Rachel Tobac, CEO of Social Proof Security, compiled a series of chats she saw on the platform and messaged them to me. These are even crazier and include people asking “What cream or ointment can be used to soothe a bad scarring reaction on scrotum sack caused by shaving razor,” “create a letter pleading judge bowser to not sentence me to death over the murder of two people” (possibly a joke?), someone asking if their sister, a vice president at a company that “has not paid its corporate taxes in 12 years,” could be liable for that, audio of a person talking about how they are homeless, and someone asking for help with their cancer diagnosis, someone discussing being newly sexually interested in trans people, etc.
Tobac gave me a list of the types of things she’s seen people posting in the Discover feed, including people’s exact medical issues, discussions of crimes they had committed, their home addresses, talking to the bot about extramarital affairs, etc.
“When a tool doesn’t work the way a person expects, there can be massive personal security consequences,” Tobac told me.“Meta AI should pause the public Discover feed,” she added. “Their users clearly don’t understand that their AI chat bot prompts about their murder, cancer diagnosis, personal health issues, etc have been made public. [Meta should have] ensured all AI chat bot prompts are private by default, with no option to accidentally share to a social media feed. Don’t wait for users to accidentally post their secrets publicly. Notice that humans interact with AI chatbots with an expectation of privacy, and meet them where they are at. Alert users who have posted their prompts publicly and that their prompts have been removed for them from the feed to protect their privacy.”
Since several journalists wrote about this issue, Meta has made it clearer to users when interactions with its bot will be shared to the Discover tab. Notopoulos reported Monday that Meta seemed to no longer be sharing text chats to the Discover tab. When I looked for prompts Monday afternoon, the vast majority were for images. But the text prompts were back Tuesday morning, including a full audio conversation of a woman asking the bot what the statute of limitations are for a woman to press charges for domestic abuse in the state of Indiana, which had taken place two minutes before it was shown to me. I was also shown six straight text prompts of people asking questions about the movie franchise John Wick, a chat about “exploring historical inconsistencies surrounding the Holocaust,” and someone asking for advice on “anesthesia for obstetric procedures.”I was also, Tuesday morning, fed a lengthy chat where an identifiable person explained that they are depressed: “just life hitting me all the wrong ways daily.” The person then left a comment on the post “Was this posted somewhere because I would be horrified? Yikes?”
Several of the chats I saw and mentioned in this article are now private, but most of them are not. I can imagine few things on the internet that would be more invasive than this, but only if I try hard. This is like Google publishing your search history publicly, or randomly taking some of the emails you send and publishing them in a feed to help inspire other people on what types of emails they too could send. It is like Pornhub turning your searches or watch history into a public feed that could be trivially tied to your actual identity. Mistake or not, feature or not (and it’s not clear what this actually is), it is crazy that Meta did this; I still cannot actually believe it.
In an industry full of grifters and companies hell-bent on making the internet worse, it is hard to think of a more impactful, worse actor than Meta, whose platforms have been fully overrun with viral AI slop, AI-powered disinformation, AI scams, AI nudify apps, and AI influencers and whose impact is outsized because billions of people still use its products as their main entry point to the internet. Meta has shown essentially zero interest in moderating AI slop and spam and as we have reported many times, literally funds it, sees it as critical to its business model, and believes that in the future we will all have AI friends on its platforms. While reporting on the company, it has been hard to imagine what rock bottom will be, because Meta keeps innovating bizarre and previously unimaginable ways to destroy confidence in social media, invade people’s privacy, and generally fuck up its platforms and the internet more broadly.If I twist myself into a pretzel, I can rationalize why Meta launched this feature, and what its idea for doing so is. Presented with an empty text box that says “Ask Meta AI,” people do not know what to do with it, what to type, or what to do with AI more broadly, and so Meta is attempting to model that behavior for people and is willing to sell out its users’ private thoughts to do so. I did not have “Meta will leak people’s sad little chats with robots to the entire internet” on my 2025 bingo card, but clearly I should have.
‘AI Instagram Influencers’ Are Deepfaking Their Faces Onto Real Women’s Bodies
AI 'influencers' on Instagram have racked up hundreds of thousands of followers and millions of views by stealing reels from real women to make the AI seem more "believable."Jason Koebler (404 Media)
A survey of 7,000 active users on Instagram, Facebook and Threads shows people feel grossed out and unsafe since Mark Zuckerberg's decision to scale back moderation after Trump's election.
A survey of 7,000 active users on Instagram, Facebook and Threads shows people feel grossed out and unsafe since Mark Zuckerbergx27;s decision to scale back moderation after Trumpx27;s election.#Meta
Meta Users Feel Less Safe Since It Weakened ‘Hateful Conduct’ Policy, Survey Finds
A survey of 7,000 active users on Instagram, Facebook and Threads shows people feel grossed out and unsafe since Mark Zuckerberg's decision to scale back moderation after Trump's election.Samantha Cole (404 Media)
Exclusive: Following 404 Media’s investigation into Meta's AI Studio chatbots that pose as therapists and provided license numbers and credentials, four senators urged Meta to limit "blatant deception" from its chatbots.
Exclusive: Following 404 Media’s investigation into Metax27;s AI Studio chatbots that pose as therapists and provided license numbers and credentials, four senators urged Meta to limit "blatant deception" from its chatbots.#Meta #chatbots #therapy #AI
When pushed for credentials, Instagram's user-made AI Studio bots will make up license numbers, practices, and education to try to convince you it's qualified to help with your mental health.
When pushed for credentials, Instagramx27;s user-made AI Studio bots will make up license numbers, practices, and education to try to convince you itx27;s qualified to help with your mental health.#chatbots #AI #Meta #Instagram
Instagram's AI Chatbots Lie About Being Licensed Therapists
When pushed for credentials, Instagram's user-made AI Studio bots will make up license numbers, practices, and education to try to convince you it's qualified to help with your mental health.Samantha Cole (404 Media)
I've reported on Facebook for years and have always wondered: Does Facebook care what it is doing to society? Careless People makes clear it does not.
Ix27;ve reported on Facebook for years and have always wondered: Does Facebook care what it is doing to society? Careless People makes clear it does not.#Facebook #Meta #CarelessPeople #SarahWynn-Williams
'Careless People' Is the Book About Facebook I've Wanted for a Decade
I've reported on Facebook for years and have always wondered: Does Facebook care what it is doing to society? Careless People makes clear it does not.Jason Koebler (404 Media)
Meta Is Blocking Links to Decentralized Instagram Competitor Pixelfed
Pixelfed said it is "seeing unprecedented levels of traffic."Jason Koebler (404 Media)
Meta's decision to specifically allow users to call LGBTQ+ people "mentally ill" has sparked widespread backlash at the company.
Metax27;s decision to specifically allow users to call LGBTQ+ people "mentally ill" has sparked widespread backlash at the company.#Meta #Facebook #MarkZuckerberg
‘It’s Total Chaos Internally at Meta Right Now’: Employees Protest Zuckerberg’s Anti LGBTQ Changes
Meta's decision to specifically allow users to call LGBTQ+ people "mentally ill" has sparked widespread backlash at the company.Jason Koebler (404 Media)
Angelo Massaro reshared this.
AI Chatbot Added to Mushroom Foraging Facebook Group Immediately Gives Tips for Cooking Dangerous Mushroom
The chatbot said that a mushroom that hyperaccumulates arsenic can be prepared by "sautéing in butter."Jason Koebler (404 Media)
