I am considering writing an open source software, which would help writers and editors in their work.
For this means, it would incorporate suggestions based on statistical analysis of English book corpora (e.g., frequency of word, or word compositions, so-called n-grams).
To this end, I would need to statistically analyse English books, this means:
- getting the text into an electronic system
- analysing the text
- incorporating the results of analysis into a database
Think of the results like available on the lotrproject about the Lord of the Rings books.
Now, of course, getting the text into an electronic system is against most disclaimers in books about copyright. However, I would argue that I am using the text for research and non-profit purposes, which according to my view passes the first factor of the four-factor balancing test.
The second factor is a harder nut to crack, as these are of course the blood and essence of copyright: works of fiction and non-fiction. However, I would argue that I will destroy the original work, I would just store statistics about it in my system, and the input I am using just for research purposes, which I will not distribute afterwards, nor read myself; I will just use it in a software for the analysis.
About factor three... I would of course need to use the whole work as input, otherwise the statistical analysis would be incorrect.
Factor four: the effect on the market value is zero or positive. I would not distribute which works were the basis of the statistics, or maybe just a list of the books which were included. The database itself would not contain a direct reference (e.g., it just would state that "in young-adult books" the phrase "c'mon mate" comes forth with this-and-this frequency, not that "in the Harry Potter books it's used n-times"). A sub-question would be, whether I'm allowed to distribute word frequencies and additional statistics about the books themselves (again cf. the above linked lotrproject page for examples).
But, of course IANAL, that's why I'm asking this question here.