Voting on Chat Control concluded on 11 March 2026 in Brussels, with the European Parliament demanding changes to the temporary Chat Control regime, which is set to expire in April 2026. The Parliament supported amendments that promote a targeted approach, restricting monitoring only to individuals under suspicion rather than permitting indiscriminate, general monitoring of all users.

Members of the European Parliament who opposed the Chat Control regime argued that the use of technology to support and mobilise child protection from online abuse should not intrude upon the privacy of users.

Markéta Gregorová said:

“Until now, this system represented a completely disproportionate intrusion into our privacy.

Platforms were scanning millions of private messages of innocent citizens without any reasonable suspicion.

Thanks to the amendment we proposed and which the European Parliament supported today, the report now clearly moves toward a targeted approach. Monitoring should only apply to the communications of suspected individuals and only with judicial authorisation. This is an important step forward for protecting Europeans’ fundamental rights.”

The positive outcome reflects the collective action of digital-rights advocates and civil society organisations, who argued that scanning all private messages would amount to mass surveillance.

The positive outcome reflects the collective action of digital-rights advocates and civil society organisations, who argued that scanning all private messages would amount to mass surveillance.

Patrick Breyer (Pirates Party) commented:

“Today is a sensational victory for the countless citizens who made calls and sent emails to save their digital privacy of correspondence. Digital privacy is alive! Just as with our physical mail, the warrantless screening of our digital communications must remain taboo. EU governments must finally realize that true child protection requires secure apps (‘Security by Design’), the removal of illegal material at the source, and targeted investigations against suspects with a judicial warrant—not overreaching, pointless mass surveillance.”

However, this victory may only be temporary. With the current regime set to expire in April 2026, the debate is likely to intensify once again. The central question remains whether the drive to protect children online will continue to be pitted against the fundamental rights of privacy, encryption, and freedom of communication.

Here, the key concern is that the technological infrastructure behind “chat control”, particularly AI-driven scanning algorithms, may still pose serious risks to digital freedoms. At this juncture, it is important to understand the mechanism of AI-driven scanning algorithms and the potential risks they pose to the user’s privacy.

The algorithmic core of chat control

At the centre of the policy debate is the growing use of artificial intelligence to detect child sexual abuse material (CSAM) online. Organisations such as Thorn have developed machine-learning systems that analyse images, videos, and text patterns to identify potentially abusive content. The organisations then aggressively lobbied the Commission to make their tools mandatory. These tools promise to detect not only known illegal images but also previously unseen material by recognising patterns associated with exploitation.

Supporters of such technologies argue that AI can dramatically accelerate investigations and help identify victims more quickly.

Here is an overview of the main AI technologies used in Chat Controls:

• Client-Side Scanning (CSS): AI tools built into a user’s device ( smartphone or laptop) that scan messages, images, or files before they are encrypted and sent or after they are received.
• Image and Video Classifiers: AI models trained to detect child sexual abuse material (CSAM) in images or videos, including content that has not been previously reported.
• Text Analysis (NLP -Natural Language Processing): Algorithms that analyse chat messages to identify possible grooming behaviour, such as attempts to manipulate minors or solicit sexual content.
• Perceptual Hashing (PhotoDNA/PDQ): Tools that create a digital fingerprint of known illegal images, allowing platforms to detect and block previously identified abuse material quickly.

The above-mentioned core technologies operate through scanning, analysis & classification, and flagging & reporting.

However, as technology advances quickly and AI takes on a bigger role in monitoring online spaces, digital rights advocates see this as a threat to user privacy. They argue that these tools could lead to automated monitoring of billions of private messages, changing how digital communication works.

This concern is not just theoretical or a political argument. Many proposals in the EU’s Child Sexual Abuse Regulation would require platforms to scan user content, including possibly encrypted messages, to find and flag suspicious material. Critics warn that these systems could turn messaging platforms into places of constant algorithmic surveillance. Instead of police focusing on specific suspects, algorithms would scan everyone’s private messages for possible wrongdoing. This could harm privacy rights and create extra work for law enforcement.

Irena Joveva, LIBE shadow rapporteur from Gibanje Svoboda (Slovenia), argued that end-to-end encryption must be upheld. At the same time, any potential scanning should remain limited to known illegal material and suspected users. She further stressed that Europe can protect children from online predators without eavesdropping on every citizen.

Such arguments reflect a broader principle in European law: that surveillance measures must be proportionate and limited to specific investigations.

​

The dangers of algorithmic detection

A few case studies:

• In 2021, Apple Inc. announced a system that would use on-device scanning to detect child sexual abuse material before photos were uploaded to iCloud. Security researchers identified flaws in the NeuralHash hashing tool. The argument was that unrelated images could produce the same hash value. This raised the possibility that harmless images might be incorrectly flagged as illegal material.
• A woman named Riley, a fitness center owner, had her business accounts on Facebook and other Meta platforms suspended following a wrongful flagging that came under the purview of violations of terms of use in relation to child sexual exploitation.
• Photographs of toddlers taken by parents to solicit medical advice and prescription have been reported by Google’s AI systems without any prior notice to the parents under child exploitation. Another example of a false alarm without investigating the connotations and situations.

The above are a few examples from millions of harrowing experiences users undergo as a result of false implications arising from AI algorithm detections. Certainly, a strong reason why digital rights advocates believe automated detection can go wrong in practice.

Even when used for legitimate purposes, AI detection systems remain controversial. Critics argue that these tools are far from reliable.

Algorithms trained to detect abusive material or suspicious conversations can generate large numbers of false positives, incorrectly flagging innocent content as suspicious. When such systems operate at a massive scale, even a small error rate could lead to thousands of wrongful reports and investigations.

Privacy advocates also warn that AI moderation systems struggle to interpret context. Automated detection tools could potentially misinterpret images, jokes, or ordinary conversations.

For activists, this raises a fundamental concern: should algorithms be trusted to judge private conversations?

Encryption under pressure

Another major concern is the potential impact of chat scanning on encrypted communication.

Many modern messaging platforms use end-to-end encryption to ensure that only the sender and recipient can view communications. Critics argue that scanning technologies, especially those involving client-side scanning, could undermine this security by analysing content before it is encrypted.

Security researchers have warned that weakening encryption for surveillance purposes could expose users to cyber threats and undermine digital security more broadly. Once a system capable of inspecting private messages exists, they argue, it becomes difficult to guarantee it will be used for only one purpose.

A wider struggle over algorithmic governance

The debate over chat control reflects a broader transformation in digital governance. Across Europe, algorithms are increasingly used to moderate content, evaluate behaviour, and monitor digital activity.

Digital rights groups warn that integrating AI-driven scanning into private communications could set a precedent for broader algorithmic surveillance. What begins as a child-protection measure could evolve into a broader monitoring infrastructure embedded in everyday digital platforms.

For this reason, activists argue that the recent parliamentary vote represents more than a procedural decision. It is a signal that lawmakers recognise the risks of indiscriminate digital surveillance.

The future of private communication

The fight over chat control is far from over. Negotiations on the EU’s long-term regulation to combat online child abuse are still ongoing, and the final shape of the law remains uncertain.

What is clear, however, is that artificial intelligence will play an increasingly central role in how online platforms detect and moderate harmful content.

The challenge for policymakers is to harness these technologies without compromising fundamental rights.

For digital rights advocates like the European Pirates, EDRI, and the European Parliament Research Services, the answer lies in maintaining a clear boundary: protecting children online must not come at the cost of turning private communication into a permanently monitored space governed by algorithms.

​

europeanpirates.eu/ai-chat-sca…