I work in information security, and one project I have been considering undertaking would involve using a bot (automated web spider) which crawls Tor hidden services for analysis. I am currently in the process of writing the code for the bot. I am concerned that I will run afoul of laws regarding child pornography or copyright infringement, as some hidden services contain such material. The current implementation of the bot does not make a distinction between legal and illegal sites. It does not query me for permission before performing an action (I never give it the "go ahead").
Some relevant information:
The bot will be running remotely on a rented server in US jurisdiction.
No web resources, legal or otherwise, will be saved to persistent storage.
I will not be viewing any material the bot accesses. I will only view its analysis.
I cannot keep a site blacklist. The bot will crawl as far as it can.
The crawling activity will not interfere with the functionality of the website.
Only public resources that can be accessed by anyone will be crawled.
The bot may attempt to sign itself up to websites if registration is necessary.
Am I responsible for any laws broken by the bot, given that I am not intending for it to access illegal material but am aware that there is a distinct possibility that it will? Is there anything I should be aware of to reduce the risk of the bot taking an illegal action which I would be responsible for? Given that one could argue (correctly, but out of context) that I am writing a program that signs itself up to illegal websites and downloads illegal material, this makes me very weary.