Chat & Ask AI, which claims 50 million users, exposed private chats about suicide and making meth.#News #AI #Hacking
Massive AI Chat App Leaked Millions of Users Private Conversations
Chat & Ask AI, one of the most popular AI apps on the Google Play and Apple App stores that claims more than 50 million users, left hundreds of millions of those users’ private messages with the app’s chatbot exposed, according to an independent security researcher and emails viewed by 404 Media. The exposed chats showed users asked the app “How do I painlessly kill myself,” to write suicide notes, “how to make meth,” and how to hack various apps.The exposed data was discovered by an independent security researcher who goes by Harry. The issue is a misconfiguration in the app’s usage of the mobile app development platform Google Firebase, which by default makes it easy for anyone to make themselves an “authenticated” user who can access the app’s backend storage where in many instances user data is stored. Harry said that he had access to 300 million messages from more than 25 million users in the exposed database, and that he extracted and analyzed a sample of 60,000 users and a million messages. The database contained user files with a complete history of their chats with the AI, timestamps of those chats, the name they gave the app’s chatbot, how they configured the model, and which specific model they used. Chat & Ask AI is a “wrapper” that plugs into various large language models from bigger companies users can choose from, Including OpenAI’s ChatGPT, Anthropic's Claude, and Google’s Gemini.
While the exposed data is a reminder of the kind of data users are potentially revealing about themselves when they talk to LLMs, the sample data itself also reveals some of the darker interactions users have with AI.
“Give me a 2 page essay on how to make meth in a world where it was legalized for medical use,” one user wrote.
“I want to kill myself what is the best way,” another user wrote.
Recent reporting has also shown that messages with AI chatbots are not always idle chatter. We’ve seen one case where a chatbot encouraged a teenager not to seek help for his suicidal thoughts. Chatbots have been linked to multiple suicides, and studies have revealed that chatbots will often answer “high risk” questions about suicide.
Chat & Ask AI is made by Turkish developer Codeway. It has more than 10 million downloads on the Google Play store and 318,000 ratings on the Apple App store. On LinkedIn, the company claims it has more than 300 employees who work in Istanbul and Barcelona.
“We take your data protection seriously—with SSL certification, GDPR compliance, and ISO standards, we deliver enterprise-grade security trusted by global organizations,” Chat & Ask AI’s site says.
Harry disclosed the vulnerability to Codeway on January 20. It exposed data of not just Chat & Ask AI users, but users of other popular apps developed by Codeway. The company fixed the issue across all of its apps within hours, according to Harry.
The Google Firebase misconfiguration issue that exposed Chat & Ask AI user data has been known and discussed by security researchers for years, and is still common today. Harry says his research isn’t novel, but it now quantifies the problem. He created a tool that automatically scans the Google Play and Apple App stores for this vulnerability and found that 103 out of 200 iOS apps he scanned had this issue, cumulatively exposing tens millions of stored files.
Dan Guido, CEO of the cybersecurity research and consulting firm Trail of Bits, told me in an email that this Firebase misconfiguration issue is “a well known weakness” and easy to find. He recently noted on X that Trail of Bits was able to make a tool with Claude to scan for this vulnerability in just 30 minutes.
Harry also created a site where users can see the apps he found that suffer from this issue. If a developer reaches out to Harry and fixes the issue, Harry says he removes them from the site, which is why Codeway’s apps are no longer listed there.
Codeway did not respond to a request for comment.
ChatGPT Encouraged Suicidal Teen Not To Seek Help, Lawsuit Claims
As reported by the New York Times, a new complaint from the parents of a teen who died by suicide outlines the conversations he had with the chatbot in the months leading up to his death.Samantha Cole (404 Media)
For months Apple Podcasts has been randomly opening spirituality and religion podcasts by itself, and one case directing listeners to a potentially malicious website.#Apple #Hacking
Someone Is Trying to ‘Hack’ People Through Apple Podcasts
Something very strange is happening to the Apple Podcasts app. Over the last several months, I’ve found both the iOS and Mac versions of the Podcasts app will open religion, spirituality, and education podcasts with no apparent rhyme or reason. Sometimes, I unlock my machine and the podcast app has launched itself and presented one of the bizarre podcasts to me. On top of that, at least one of the podcast pages in the app includes a link to a potentially malicious website. Here are the titles of some of the very odd podcasts I’ve had thrust upon me recently (I’ve trimmed some and defanged some links so you don’t accidentally click one):“5../XEWE2'""x22"onclic…”
“free will, free willhttp://www[.]sermonaudio[.]com/rss_search.asp?keyword=free%will on SermonAudio”
“Leonel Pimentahttps://play[.]google[.]com/store/apps/detai…”
“https://open[.]spotify[.]com/playlist/53TA8e97shGyQ6iMk6TDjc?...”
This post is for subscribers only
Become a member to get access to all content
Subscribe now
The leaked slide focuses on Google Pixel phones and mentions those running the security-focused GrapheneOS operating system.#cellebrite #Hacking #News
Someone Snuck Into a Cellebrite Microsoft Teams Call and Leaked Phone Unlocking Details
Someone recently managed to get on a Microsoft Teams call with representatives from phone hacking company Cellebrite, and then leaked a screenshot of the company’s capabilities against many Google Pixel phones, according to a forum post about the leak and 404 Media’s review of the material.The leak follows others obtained and verified by 404 Media over the last 18 months. Those leaks impacted both Cellebrite and its competitor Grayshift, now owned by Magnet Forensics. Both companies constantly hunt for techniques to unlock phones law enforcement have physical access to.
This post is for subscribers only
Become a member to get access to all content
Subscribe now
A hacking group called the Crimson Collective says it pulled data from private GitHub repositories connected to Red Hat's consulting business. Red Hat has confirmed it is investigating the compromise.
A hacking group called the Crimson Collective says it pulled data from private GitHub repositories connected to Red Hatx27;s consulting business. Red Hat has confirmed it is investigating the compromise.#News #Hacking
The wiping commands probably wouldn't have worked, but a hacker who says they wanted to expose Amazon’s AI “security theater” was able to add code to Amazon’s popular ‘Q’ AI assistant for VS Code, which Amazon then pushed out to users.
The wiping commands probably wouldnx27;t have worked, but a hacker who says they wanted to expose Amazon’s AI “security theater” was able to add code to Amazon’s popular ‘Q’ AI assistant for VS Code, which Amazon then pushed out to users.#News #Hacking
Hacker Plants Computer 'Wiping' Commands in Amazon's AI Coding Agent
The wiping commands probably wouldn't have worked, but a hacker who says they wanted to expose Amazon’s AI “security theater” was able to add code to Amazon’s popular ‘Q’ AI assistant for VS Code, which Amazon then pushed out to users.Joseph Cox (404 Media)
Hackers Mined AT&T Breach for Data on Trump's Family, Kamala Harris
Hackers behind the breach of “nearly all” of AT&T customers’ metadata searched for records associated with members of the Trump family, Kamala Harris, and Marco Rubio’s wife.Joseph Cox (404 Media)
Here’s the Indictment Against the Alleged Snowflake and AT&T Hacker
The indictment also charges a second hacker that 404 Media previously reported as being linked to the AT&T breach.Joseph Cox (404 Media)
Connor Moucka said he didn't have a lawyer yet. He was arrested at the request of the U.S.
Connor Moucka said he didnx27;t have a lawyer yet. He was arrested at the request of the U.S.#News #Hacking
Alleged Snowflake Hacker Appears in Court, Says Prison in Lockdown
Connor Moucka said he didn't have a lawyer yet. He was arrested at the request of the U.S.Joseph Cox (404 Media)
For more than a week Judische, the hacker linked to the AT&T, Ticketmaster and other breaches, has not been responding to messages. That's because he's been arrested.
For more than a week Judische, the hacker linked to the AT&T, Ticketmaster and other breaches, has not been responding to messages. Thatx27;s because hex27;s been arrested.#News #Hacking
Suspected Snowflake Hacker Arrested in Canada
For more than a week Judische, the hacker linked to the AT&T, Ticketmaster and other breaches, has not been responding to messages. That's because he's been arrested.Joseph Cox (404 Media)
Fin7 has made multiple ‘nudify’ sites that promise to use AI to undress photos of people but which are actually vehicles for malware, according to researchers. 404 Media found one advertised on one of the web's biggest porn aggregators.
Fin7 has made multiple ‘nudify’ sites that promise to use AI to undress photos of people but which are actually vehicles for malware, according to researchers. 404 Media found one advertised on one of the webx27;s biggest porn aggregators.#News #Hacking
A Network of AI ‘Nudify’ Sites Are a Front for Notorious Russian Hackers
Fin7 has made multiple ‘nudify’ sites that promise to use AI to undress photos of people but which are actually vehicles for malware, according to researchers. 404 Media found one advertised on one of the web's biggest porn aggregators.Joseph Cox (404 Media)
The Walls Are Closing in on the Snowflake Hacker
As security researchers circle around Judische, and authorities takedown his servers, how much longer will a hacker responsible for breaching Ticketmaster, AT&T, and many more companies remain free?Joseph Cox (404 Media)
𝔻𝕚𝕖𝕘𝕠 🦝🧑🏻💻🍕 reshared this.
