1

Simple question: I want to get specific data, but the API does not have the endpoint I want, so what I want to do is scrape the database for the resources and then use my own filtering to retrieve and display the data part I want. I really don't want to get sued. Any insight would be very much appreciated.

Improve this question

2 Answers 2

Reset to default
1

As with any question dealing with accessing, copying and repurposing data, copyright law comes into play. Read I have a question about copyright. What should I read before I ask it? :

Copyright law protects a creator’s right to control certain intangible works that they have created, specifically protecting original expression but not underlying facts or ideas. Modern copyright law is based on the Berne Convention, and in the US, the statutory basis for copyright law is Title 17 of the US Code. 17 USC 102(a) states the basic rule regarding what is protected: original works of authorship fixed in any tangible medium of expression, now known or later developed, from which they can be perceived, reproduced, or otherwise communicated, either directly or with the aid of a machine or device

That means the data on the site is under copyright.

Now, permission to scrape, use and repurpose the copyrighted data from that site - which is what you want to do - will be covered in the site Terms of Service (TOS). Look for that at the website and read it.

What does the TOS say about accessing and using the data via the API? What are the conditions for data access and usage that the TOS outlines? Does the TOS say you can only access the data; or does it also say you can filter and use their data and display the result?

All those points - and more - should be clearly outlined; if you "don't want to get sued", read the TOS and be sure to comply with its conditions.

It doesn't really matter if the API endpoint is not known or documented; in fact, the TOS may forbid access to data by undocumented endpoints or by any other method other than the API. Read it.

If in fact the TOS does not say anything about accessing with undocumented APIs, ask the site owners if you can access the data by your method. It's much better to go that route and find out, rather than do it anyway, have them find out by log entries, change the TOS to forbid that access, and track you down.

See earlier questions on LSE: Is it legal to scrape a website and create my own database? and Is scraping real estate data illegal in anyway? and Can I legally scrape data from a website for use in my app?

Improve this answer
-1

I assume by "scraping", you mean reading someone's web site and parsing the information in the web page.

Normally that is entirely legal.

If information is published on a public web page, there is no reason why you can't read it. Just because the form of information is inconvenient or requires more programming to read the information is legally irrelevant.

Improve this answer
2
  • I see what you're saying, but where do we draw the line? If I gather data by changing a GET parameter? An undocumented POST parameter? SQL injection?
    – Darren H
    Commented Jan 23, 2017 at 0:47
  • Id like to add that the downvote wasn't me
    – Darren H
    Commented Jan 23, 2017 at 0:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .