Is it allowed to aggregate social media data?

Question

We are a group of data scientists, trying to build a social media search engine that basically focuses on trends and follows the method of store->index->rank method of storing social data from Twitter, Instagram, Vine, Tumblr (for now; we plan to add more in the future).

We have researched Terms of Use of major social networks, and we haven't found anything substantial that it is unlawful to aggregate social media data.

So, does someone know or experienced such a use case before that we should be wary of?

Answer 1

I'm not a lawyer but pretty sure this is illegal in some countries which is problematic since you said that you plan to operate globally in the future.

The first subparagraph of § 87b Urheberrechtsgesetz in Germany is:

Der Datenbankhersteller hat das ausschließliche Recht, die Datenbank insgesamt oder einen nach Art oder Umfang wesentlichen Teil der Datenbank zu vervielfältigen, zu verbreiten und öffentlich wiederzugeben. Der Vervielfältigung, Verbreitung oder öffentlichen Wiedergabe eines nach Art oder Umfang wesentlichen Teils der Datenbank steht die wiederholte und systematische Vervielfältigung, Verbreitung oder öffentliche Wiedergabe von nach Art und Umfang unwesentlichen Teilen der Datenbank gleich, sofern diese Handlungen einer normalen Auswertung der Datenbank zuwiderlaufen oder die berechtigten Interessen des Datenbankherstellers unzumutbar beeinträchtigen.

If you care fore the details of a law that might cause you problems, you can use Google Translator on it.

However, I'll briefly state the most important facts (be aware that this is German law and is only held as an example of a jurisdiction which might cause you some trouble):

• The one who created the database has the exclusive right to do the usual copyright stuff.
• As someone else you aren't allowed to copy a substantial part of it without permission of the copyright holder.
• If you repeatedly copy parts of it, that's considered copying it. (Only if that isn't considered normal use of the database or violates justified interests of the database producer.)

Since you plan to become a competitor in the same market as the ones who's databases you partially copy, this probably violates justified interests of the copyright holders and furthermore copying data from social networks probably isn't considered normal use.

§ 87a Urheberrechtsgesetz subparagraph 1 states (this is only part of it):

Datenbank im Sinne dieses Gesetzes ist eine Sammlung von Werken, Daten oder anderen unabhängigen Elementen, die systematisch oder methodisch angeordnet und einzeln mit Hilfe elektronischer Mittel oder auf andere Weise zugänglich sind und deren Beschaffung, Überprüfung oder Darstellung eine nach Art oder Umfang wesentliche Investition erfordert.

From which (again, use Google Translator) you can take what qualifies as a database. Basically it's a collection of works, data, or other stuff, which took a substantial investment (of whatever, that's not stated there) to acquire, check, or represent. You can be pretty sure that's the case for the content of about every commercial website.

Answer 2

As long as this social media data analytics falls under OSINT (Open Source Intelligence), meaning only public available data, it should be okay.

Plenty of other companies do exactly this already and make money with it, without getting sued, so I do not see any problem with it. Example: http://www.vocativ.com/about/