Is Apple scanning user content for child abuse a violation of GDPR?

Question

According to this article Apple is scanning all the photos users have on their devices to detect possible child abuse. The article explains that a NeuralHash of each image will be uploaded to Apple servers. It also claims that the NeuralHash is only a fingerprint of the image. But that is not true, a hashing made by a neural network is reversible, the reconstructed image will not be as good as the original, but it can be compared to a lossy compression with high loss.

Is this compatible with the European GDPR? Can a corporation upload user content to their servers without asking the user permission?

Related question:
https://politics.stackexchange.com/questions/68507/can-countries-prevent-foreign-corporations-like-apple-from-acting-as-private-pol

Update:

Thanks to the pointer from @Fizz I found the reference to the exception for child abuse recently voted. It is a bit ambiguous because as a law it should be immediately applicable, but member states should anyway update their legislation to avoid conflict. And the news reporting the laws mention some constraints.

https://ec.europa.eu/commission/presscorner/detail/en/IP_20_2463

https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52020PC0568

Answer 1

According to a Politico article from July this year

EU Parliament lets companies look for child abuse on their platforms, with reservations

The European Parliament on Tuesday approved a controversial law that would allow digital companies to detect and report child sexual abuse on their platforms for the next three years.

Tuesday's vote was the final hurdle for the bill, and will allow companies to scan their platforms for explicit material without fear of violating Europe's strict privacy laws. The bill pitted the European Commission, who proposed the bill, and children's rights activists against the Parliament and Europe's privacy regulators, who fear the bill could undermine the EU's privacy rules.

The results showed 537 MEPs voted in favor of the bill, with 133 against and 24 abstaining. Despite the result, [some] European lawmakers warned that the rules are "legally flawed" and could crumble in front of a court. [...]

MEPs also said that the blanket scanning of private messages of European citizens to look for evidence of child grooming could clash with another set of privacy rules protecting personal data, the GDPR.

To allay the Parliament's concerns, EU countries agreed to modify the Commission’s law to add additional safeguards, including bringing in Europe's network of privacy watchdogs to advise on what technologies should be used to do the scanning, and how they should be used. They also left out audio messages from the bill. The changes prompted Sippel, who was negotiating on behalf of the Parliament, to sign off on the bill.

So YMMV, but insofar it looks pretty legal unless a court decides otherwise.

It seems Apple went public with their plan after this EU law was passed, so they probably took it into account.

---

For what's that worth, there's an analysis from someone at Cambridge (in the Compliant and Accountable Systems Group, Department of Computer Science and Technology) that:

EU law would require that Apple obtain the consent of individual iPhone users for on-device scanning. This consent would need to be opt-in, rather than opt-out; there would need to be a real possibility for users to refuse consent; users’ access to iCloud could not be made conditional on giving consent to CSAM Detection; and users must be able to withdraw consent without suffering loss of iCloud service. This may place a welcome brake on the deployment of on- device CSAM detection in the EU. However, these barriers may be removed by future EU or Member State legislation – just as similar potential barriers for automated CSAM detection by certain messaging services have already been removed by EU legislation.

So it seems that there would be a way for Apple to do this "device scanning" even in the EU with user (clickwrap) agreement...

The paper enumerates the things than can be made implicit (bundled) in a GDPR agreement, and then goes on to argue that probably none of these apply to Apple's CSAM on device, and so they'll probably need a separate checkbox for CSAM... which the paper's author is pretty sure the users would not check.

Importantly, because consent must be specific, consent to one purpose (such as processing to detect CSAM) can’t be presumed from consent to another purpose (such as processing for cloud backup). Nor can consent to processing to detect CSAM be ‘bundled’ with consent to cloud backup. GDPR strongly indicates that ‘bundling’ – the practice of making access to a service conditional on giving consent to processing that is not necessary for that service – is not permitted. [...]

It is unlikely in the extreme that even minimally informed holders of CSAM would give consent to Apple’s CSAM Detection system.

The paper more tenuously (IMHO) argues then that Apple needs do the same for their server-side scanning in the EU, i.e. get explicit consent, because it's somehow tied to the on-device scanning via the iCloud account. (The author also makes their personal disapproval of CSAM in all forms more explicit towards the end of the paper.) But still that someone who disapproves of CSAM entertains that it might be deployed via clickwrap user consent (albeit with a separate checkbox) is noteworthy.

In this context I'll note (even though the paper doesn't), that there is a 2019 CJEU decision that pre-checking certain kinds of checkboxes (like for "nonessential" cookies) is not legal. I suspect CSAM will fall in this kind of category... so they won't be able to have the checkbox for it pre-checked, unless Apple does something more devious and make it so that image hashes become "essential" to something else that the user would more readily like to agree to...

Also, looking at a current (August 1) Apple description of their system, it seems they only plan to run the image hashing on the device (and upload a threshold-encrypted version of the result, so that only multiple, cumulative "hits" on several images would be detectable/decryptable on the server), which would basically make their system (as currently envisaged) inoperable without the local hash step. I've also looked at Commission's 70-page impact assessment for their most recent proposal (the one discussed at the beginning of this answer), but it only discusses things like PhotoDNA and server-side hashing. So I guess they were unware that someone might plan to do [only] client-side hashing as part of a CSAM design.

If this goes to an EU court, I guess an issue will be when and why iOS would create those (threshold-encrypted) vouchers that contain the images hashes, e.g. if they'd be considered an essential part/step of an otherwise approved purpose...

Answer 2

Apple's CSAM scanning techniques involve a number of different approaches. Some of these could violate European privacy laws, some will not. However, Apple is not currently planning to field their new developments like NeuralHash in the EU, so the matter is hypothetical.

CSAM scanning in European privacy law

The GDPR is the foundation of European data protection law. It requires that all processing of personal data has a clear legal basis, and prohibits processing of “special categories” of personal data in most cases (e.g. biometrics, or information regarding sexual orientation). The GDPR is particularized and complemented by the ePrivacy directive, which lays down more specific rules for electronic communications services and information society services (e.g. websites and apps). This involves details on the confidentiality of communications, and on the use of traffic data.

Under this regime, some service providers have performed server-side CSAM scanning (e.g. by providers of emails or chat services) under the theory that they are not bound by confidentiality of communications, that they have a legitimate interest to do so under GDPR, and that no special categories of data are impacted. But with Directive 2018/1972, the legislature has clarified that “number-independent interpersonal communications services” (such as chat services or emails) are also subject to confidentiality of communications in the sense of the ePrivacy directive, making such scanning illegal: there is no EU-level legal basis for violating confidentiality of communications.

Recent derogations

Individual EU member states had already issued derogations from ePrivacy to allow CSAM scanning to continue. The EU parliament also passed Regulation 2021/1232 that provides certain exceptions from the ePrivacy requirements, in particular from Art 5(1) and 6(1) ePrivacy.

• Art 5(1) is about confidentiality of communications and prohibits:

  listening, tapping, storage or other kinds of interception or surveillance of communications and the related traffic data by persons other than users

• Art 6(1) is about erasing traffic data once it's no longer needed for the communication:

  Traffic data relating to subscribers and users processed and stored by the provider of a public communications network or publicly available electronic communications service must be erased or made anonymous when it is no longer needed for the purpose of the transmission of a communication

This means that a communication service provider can intercept, store, and surveil communications and related traffic and store traffic data as strictly necessary in order to detect, remove, and report CSAM.

Apple's proposed techniques are only partly covered by the exemptions

Some of Apple's proposed methods are clearly covered by this derogation. For example, Apple is proposing to scan incoming messages sent to minors.

But Apple is also proposing to scan images on an end user's device before these are part of a communication. This is problematic:

• The exemption in Regulation 2021/1232 only covers communications and traffic data, not data at rest on a device.
• In particular, the exemption is scoped to Art 5(1) and 6(1) of the ePrivacy directive.
• Data at rest is covered by Art 5(3) of the ePrivacy directive. This section is not covered by the EU-wide exemption.

So it seems that Apple's more contentious scanning approaches could be a violation of European privacy laws. It is therefore understandable that Apple is only planning to deploy it's new CSAM scanning in the US initially, and then expand to other jurisdictions as appropriate. While there is no EU-wide legal basis for scanning files on a user's device, it is possible that individual EU member states have issued laws that authorize such searches.

The matter is a bit more complex, since Apple doesn't search all images on the user's device - they are merely planning to require that images uploaded to their backup service are accompanied by a matching safety token that can be used for server-side CSAM checking. The token is computed on the user's device, but the token is arguably necessary to provide a service requested by the user. That can be argued to comply with ePrivacy and with the GDPR. If server-side CSAM checking is legal (it is) and Apple's approach is more privacy-preserving than a server-side scan (it probably is), then it would even be possible to argue that Apple's architecture is required by the GDPR.

So there are a variety of nuanced opinions to be had about this matter. My personal opinion is that as proposed, Apple's solution is technically brilliant but problematic on a policy level: by offloading a check from their servers to the end user's devices, Apple is whitewashing their products without actually improving privacy – it would be very different if they were using this system to detect CSAM in otherwise end-to-end encrypted backups. So far, this seems like an unnecessary intrusion into the user's device, and I believe this intrusion fails to comply with Art 5(3) of the ePrivacy directive.