2

I have read several news in the EFF's website about court appeals and fights related to web scraping.

I am a programmer and have always been worried about what are the limits of scraping.

Normally, I think any public site can be scraped, as 'viewing' a website is not different from that.

That seems to be the EFF interpretation too. And the terms of service seem not to count, as far as I understand.

I would like insight in the following situations:

  • What if we scrape using a username/pwd?
  • What rule applies when the 2 parties are in different countries?

2 Answers 2

1

The U.S. 9th Circuit Court of Appeals ruled on September 9 that the U.S.'s Computer Fraud and Abuse Act (CFAA) is not violated when a company scrapes public websites.

Though, I suspect that scraping identity or access codes when access was not granted to you could be considered theft.

Also, see: Is it legal to share a script that scrapes a website and presents in a different format?

I've always enjoyed programming scrapes...gratifying and relaxing, like sudoku or Tetris. :-D

-shelDev

1

The CFAA is filled with words and phrases like "intentionally", "knowingly", "with intent", which allows web scrapers that play by the rules

Search engine crawlers are also web-scrapers, with some extra steps. In general, search engines respect what's in the robots.txt. That generally means they do not use username/passwords to scrap.

Most websites want you to scrap them - that's how Google works. What they don't want is you making too many requests per minute, which takes computing power away from humans, or accessing personal user information, which is why scrapers don't use username/passwords regularly.

The CFAA explicitly mentions unauthorized access

(1) having knowingly accessed a computer without authorization or exceeding authorized access, and by means of such conduct having obtained information that has been determined by the United States Government pursuant to an Executive order or statute to require protection against unauthorized disclosure for reasons of national defense or foreign relations, or any restricted data, as defined in paragraph y. of section 11 of the Atomic Energy Act of 1954, with reason to believe that such information so obtained could be used to the injury of the United States, or to the advantage of any foreign nation willfully communicates, delivers, transmits, or causes to be communicated, delivered, or transmitted, or attempts to communicate, deliver, transmit or cause to be communicated, delivered, or transmitted the same to any person not entitled to receive it, or willfully retains the same and fails to deliver it to the officer or employee of the United States entitled to receive it;

So stealing a password is illegal, but the CFAA goes further. Even using a legit password for unauthorized purposes is illegal.

The case of most interest is probably U.S v Aaron Swartz. A Harvard faculty member a with valid JSTOR login created a scraper to download all the JSTOR content. In order to get around scraper protections on JSTOR, he plugged into a data closet directly and did some other technical wizardry. He was charge by the government with felonies and committed suicide.

In this case, he had a legal username/password but wasn't allowed to scrap the site. The government felt he exceeded his access authority and charged him, even though he had authorization to access the site.

Your other question about people in 2 different countries is very easy and completely unrelated. The law that applies is the country where the computer running the scraper is located. That's why it's next to impossible to prosecute the hackers that hijack facebook accounts and push forex scams - they're not in America.

5
  • Why wasn't he allowed to scape the site, how is that line drown? also, 2nd Q isnt unrelated. It is about the limits and scope of CFAA.
    – Mah Neh
    Apr 30, 2022 at 13:08
  • @MahNeh - follow the links and you can read all the reasons the government decided to charge him. 2nd question is unrelated because any law congress passes only applies in America. They can’t pass laws governing what you do in other countries. Apr 30, 2022 at 19:14
  • Ok, I am not American. Hope you can understand that some of us live elsewhere.
    – Mah Neh
    Apr 30, 2022 at 19:26
  • All laws in America only apply to people on U.S. soil. Just like all laws in the U.K. Only apply to people in the U.K. It has nothing to do with the CFAA. You have to follow whatever the laws are in your country. Apr 30, 2022 at 20:24
  • You prosecute people elsewhere, so I got confused.
    – Mah Neh
    Apr 30, 2022 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.