Dispatches from the future The first year of Chat Control

EU law begins forcing Big Tech to tackle child abuse imagery, activist takes aim

Brussels — The Child Sexual Abuse Regulation went into effect today in the European Union. The law is seen as the EU enlisting an unexpected partner: the big tech companies it fined previously for not being private enough.

Industry adoption of end-to-end encrypted messaging technology allowed businesses and governments to communicate digitally with less concern about data leaks. The security came with a tradeoff of increased difficulty for law enforcement to investigate potential crimes.

End-to-end encryption works in part by the sender and recipient exchanging unique keys to decrypt each other’s messages. Introducing an additional decryption key for a third-party is possible and could allow law enforcement to read intercepted messages. However, the challenge of securing additional keys from also being used by adversaries prevented the technique from being used.

EU legislators’ opposition to the proposal relaxed when a different technical implementation was identified. The regulation requires messaging apps to verify photos and videos do not contain sexually explicit images involving a child before being sent encrypted. Apps can upload the unencrypted media temporarily to their moderation servers or use on-device machine learning models (AI) trained on illegal content previously obtained by law enforcement. Messaging apps must submit newly identified images to law enforcement.

Over 100 human rights organizations opposed the “chat control” proposal before it became law. Critics vowed to take the fight to the European Court of Justice for violating the right to privacy guaranteed by the Charter of Fundamental Rights of the European Union.

“In the general scanning of communications, frequent false positives cannot be avoided, even if accuracy rates are high, thereby implicating numerous innocent individuals. Given the possibility of such impacts, indiscriminate surveillance is likely to have a significant chilling effect on free expression and association, with people limiting the ways they communicate and interact with others and engaging in self-censorship,” said the UN Office of the High Commissioner for Human Rights.

“This regulation handles every photo we share with friends as potential evidence of our guilt until we have proven our innocense beyond a reasonable doubt. Today, the state surveillance of our photos is justified by child safety. Tomorrow, the scanning could expand to everything we do on our devices justified by terrorism or organized crime,” said activist and software developer Jeremiah Lee.

This story is fictional. It was never released by Reuters. See disclaimer.

“Communication chaos” for government officials as EU chat control law goes live

Brussels — Midnight negotiation led by the European Commission failed to convince the Signal Foundation to continue providing service to European members of government, security agencies and military exempted from the chat control scanning regulation.

The Commission recommended use of Signal for communication between staff and people outside the institution in 2020 after thousands of diplomatic cables were accessed illegally. It now is evaluating forking the open source project and products using the Matrix protocol, but does not expect to have a new recommendation for at least 6 months.

This story is fictional. It was never released by Reuters. See disclaimer.

Newsroom

• WhatsApp
• Messenger
• Instagram

Enabling Safer Chat for Everyone

Menlo Park — Meta continues its investment in public safety with industry-leading AI to accurately identity illegal content within the EU and report violations to law enforcement as required.

Keeping young people safe online has been a challenge since the Internet began. As criminals evolve their tactics, we have to evolve our defenses too.

We’ve trained AI using millions of nuanced moderation decisions made by humans. New regulation allows us to extend this protection to the images and videos sent encrypted between people.

The spread of Child Sexual Abuse Material (CSAM) challenges all communication platforms and cannot be solved by a single company. “Meta welcomes a more active role by governments and regulators in updating the rules for the Internet, so we can preserve what’s best about it—the freedom for people to express themselves and for entrepreneurs to build new things—while also protecting society from broader harms,” Mark Zuckerberg, Meta CEO, said.

WhatsApp connects hundreds of millions of people in the European Union and billions globally. People do amazing things with our platforms and we are committed to ensuring a positive experience for everyone.

Meta has invested more than $20 billion since 2016 in trust and safety initiatives. “Meta goes above and beyond to make sure that there are no portions of their network where this type of activity occurs,” said National Center for Missing and Exploited Children.

This press release is fictional. It was never released by Meta. See disclaimer.

The Big Story

Post-privacy era?
Not on these geeks’ watch.

In the wake of the EU’s reversal on privacy, third-party tools have restored security features forcefully removed from their favorite messaging apps, while radical new apps aspire to be regulation-proof.

A year after Chat Control, EU lawmakers debate level of surveillance needed for a safer society

Brussels — The European Union’s Child Sexual Abuse Regulation, commonly referred to as chat control, went into effect a year ago. Critics and proponents of the regulation agree on its limited success in protecting children. The next step remains a divisive debate.

Reports to law enforcement of suspected abuse imagery increased 600% in the last year when messaging apps implemented the regulation’s required automated reporting of suspicious content. The increase in abuse reports did not result in an increase in charges or convictions. Only 4 criminal charges used evidence obtained from the automated submissions and all of the investigations were underway prior to the additional evidence.

“We’ve been overwhelmed with automated reports. We don’t have the budget to hire and train enough people to review the reports quickly. It feels like social networks just outsourced their moderation problem to the government,” said a law enforcement officer requesting anonymity.

The EU Centre overseeing the regulation admitted most reports were either harmless, such as vacation photos of nude children playing at a beach, or a legal gray area of teenagers sending intimate, but consensual, photos of themselves.

Critics of chat control who feel the law went too far in restricting privacy cite the lack of increased convictions as a reason to repeal a misguided law, while proponents cite the failure as proof the law did not go far enough in aiding law enforcement.

“Crime moves fast and we must move faster. We suspect criminals now use ‘black bubble’ messaging apps to avoid interception,” said another law enforcement officer requesting anonymity.

Messaging apps using peer-to-peer technology inspired by Tor’s “dark web” became known as “black bubble” apps, a reference to Apple’s visual treatment of displaying messages sent using iMessage with a blue background and messages sent using older SMS technology to Android users with a green background.

Member states in the Council of the EU last week debated an expansion of surveillance capabilities beyond messaging apps and child abuse investigations. First proposed during Sweden’s EU Presidency in spring 2023, the updated regulation could require software service providers give law enforcement access to unencrypted user data when ordered by a judiciary.

Human rights organizations opposed the proposal, citing technical challenges in implementing such a policy and concerns about abuse.

“So-called ‘lawful access by design’ is ‘insecurity by design.’ There is no way to add backdoor access to encryption or an on-device snitch for police that cannot also be potentially exploited by a hostile nation state or criminal hacker. This is not a privacy fetish. It’s a technical reality,” said Jeremiah Lee, a data privacy engineer.

Debate over the balance between the human right to privacy and government’s ability to provide justice to victims is likely to continue for generations. For some, the debate is a distraction from investigating non-technical solutions to non-technical problems.

“What we’re talking about is a kind of self-negating paradox. You cannot do mass surveillance privately, full stop,” said Meredith Whittaker, Signal President.

This story is fictional. It was never released by Reuters. See disclaimer.