Suppose someone develops a chat app where you can send messages and pictures. Messages are stored in a database.
The developer, who is also the site owner, would have pretty much access to every messages and pictures sent by the app, via the DB console.
What if one day there are plenty of underage nudes sent everyday? How can the developer protect himself from this?
Can a developer avoid civil and especially criminal liability in such a case?
Disclaimer:
I'm not directly knowledgeable about which particular laws associate with this kind of issue, but I am reasonable aware of certain best practices and approaches that people take to try and accommodate these issues. Hopefully some sort of answer is better than nothing.
Storage concerns:
For the most part, the long-term storage of any kind of media has all sorts of risks associated with it. You mention texts and photos, but this can expand out to videos or even files in general. Although your example concern is of underage explicit photos, that is only one of many things that could be deemed a problem. Even underage photos that weren't explicit may be considered illegal without the uploader having their parent or guardian's consent. Similarly if the photo was inciting violence/terrorism, showing trade secrets, or even including brand logos, they're all viable issues that someone out there might complain about. As you can't just run each file through a photo identifier to know exactly what it is, you need to accommodate finding these problematic files in a different manner.
Possible approach:
The most common approach I've seen is to include a "Report" feature within your app. Much like a DMCA takedown notice, just because someone uploaded an illegal file to your app doesn't inherently make you a criminal; you just need to act on its removal in a timely manner. Having users report this kind of media will help you sift through most problematic files. I'm not sure, but you may also need/want some sort of contact information as well (e.g. email, phone and/or PO box to write complaints to).
I'm not sure how fast is timely and I'm not sure whether it is typical for a company to actually delete the item or to just store it separately with some sort of incident identifier. My thought process here is that the police may need evidence if they do find the person who provided such a file and they're actually pursuing them criminally. Similarly, I don't know if it's required to report certain issues to the police or if that's up to the company to follow through with. Depending on your app's growth, handling user's reports could mean you need other people (yes, possibly staff) to help stay ahead of the workload.
It's also important to note that the country you're in and the country the uploader is in could differ. Although their upload could be legally acceptable in their country, if it's not in yours then you still need to act on it because they are using your app and accepted your terms of service. These are all things that an actual lawyer would probably need to clarify, of which I am not one of those people.
