Suppose a company is developing software, and an engineer/coder uses some code generated by Code LLaMA or some similar generative AI.

Microsoft has brought out copilot, which does a similar thing and has said that they will cover any copyright claims, but even this may not be so clear.

The fundamental behavior of a generative LLM is that it produces a token at a time, rather than copying from somewhere else, and the onus would still be on the engineer/coder to ensure that the generated code is fit for purpose.

There are tools that check for copyright, but they typically check against a hash function, meaning that a single space would not match, whereas that wouldn't be enough of a change for a copyright claim, to the best of my knowledge.

So what is the way to think about the risks of using a tool like this in a company?

On the one hand, no information has been directly copied from anywhere. Code has been translated into weights, and then weights are translated into code.

On the other hand, companies developing core intellectual property are naturally sensitive to any potential claims against it.

  • 3
    Note that Copilot, GPT, and others have repeatedly been caught copying entire functions or classes verbatim. So, while your description is technically correct that they produce one token at a time, what they produce can still be a complete copy of a copyrighted work, provided the problem domain is narrow enough. Nov 29, 2023 at 18:39
  • @jorgwmittag, do you have any sources for that? I don't disbelieve you. It would also be interesting to understand what the training set is then. Like if the model was trained on open source code, then you could argue that it couldn't have copied, even if it was verbatim. I don't know if that would be a defense.
    – Dr Xorile
    Nov 29, 2023 at 20:31
  • "Like if the model was trained on open source code, then you could argue that it couldn't have copied" – This makes no sense. Whether or not you copied depends on … whether or not you copied, not what the license is. Nov 30, 2023 at 19:12
  • I meant it couldn't have copied copyrighted material. If it didn't have access to copyrighted material, it couldn't have copied copyrighted material.
    – Dr Xorile
    Dec 2, 2023 at 1:55
  • 1
    Yes, of course. Open Source relies on copyright. How can you have an Open Source copyright license if there is no copyright? Dec 2, 2023 at 20:25

3 Answers 3


The first way to look at the question is whether the output of the program is “substantially similar” to that of a protected work. A copyright infringement claim does not have to show video of a person copying the protected work, the plaintiff simply has to prove that they own the right to the original work and that the infringing work is similar to it (there are various adjectives used such as ‘probative’, ‘striking’, ‘substantial’, and their definitions and application differ according to jurisdiction). The proof requirement is not that you have to prove that the infringing work is exactly verbatim identical to the original, there is just a level of “close enough”, since the prohibited act is copying, not exact copying. If the output looks like it was copied, the defendant is likely to be found to have infringed, unless you can somehow prove that the AI output only coincidentally recreated the original. You claim that “The fundamental behavior of a generative LLM is that it produces a token at a time, rather than copying from somewhere else”, which I don’t dispute, but this is not a self-evident truth, it is only a part of a defense that has to be proven. This is a substantial risk (just saying it over and over again will not persuade a juror, when they are also being told the contrary).

As I suspect you know, you can pair a text with a word-frequency table and write a mathematical algorithm that exactly reduces any original text to a “model”, which can then exactly reproduce an original text not by copying from the clear text, but via the mathematical model. If that is what the program does, there is no question that this is illegal creation of a derivative work (in the same way that it is infringement to read a novel and start translating it into French, without the author’s permission). This is another legal risk, one that is harder to disprove, that you created an unauthoriized derivative work.

There are word-frequency tables ‘out there’ which are created by copying (infringing) protected works and reducing individual words to counts, e.g. “the” occurs 129,465,324 times. Nobody holds copyright on a single word, or two words, or three words… there is a line there, the law has not said what the bright line is (in terms of numbers of consecutive words). But one may benefit from a fair use argument, which is why at least so far nobody has been sued for creating a word frequency table, even though the act of creating the table requires copying of protected text (unless one is doing a table of frequencies for 19th century English, or 20th century laws passed by Congress). LLM works not just in terms of individual words in isolation, it encodes context, which if long enough could be very much like just the approach of substituting frequency-rank for the actual word.

A possible mitigating factor would be the “there are only so many notes” defense that sometimes figures into music infringement. There are (non-trivial) natural limits on musical compositions (fewer atoms that can be combined), because of which one may independently create a work that strongly resembles another protected work. For actual literary texts, the coincidence argument is easy to disprove, but computer code is much closer to musical compositions in that for example there are only so many ways that one can reasonably implement a bubble sort.

In theory, the creators of the language model could be found liable for copyright infringement. The main risk, in my opinion, is that the law (in the US) prohibits creation of substantially-similar text, and a user might well end up in court because they relied on a tool that can produce extremely similar works. The user does not have access to the guts of Open AI or other such LLM software, and to mount a successful defense, they will need non-trivial assistance from the creators of the model to prove that this isn’t just plain old copying.

  • Thanks for your thoughtful comments. Another issue (which perhaps I should put into my original question) is that it is difficult to see how someone would discover that closed-source code was infringing a copyright. To continue your music analogy, it's like someone is writing music in their home, without it being published or shown to anyone. Like if the music you write changes the color of your house in a non-specified way, then it's hard to imagine any copyright holder for the music having a claim.
    – Dr Xorile
    Dec 2, 2023 at 2:05

All generative AI produces either copies or derivative works

Definitions vary but in the USA 17 U.S. Code § 101 says:

A “derivative work” is a work based upon one or more preexisting works, such as a translation, musical arrangement, dramatization, fictionalization, motion picture version, sound recording, art reproduction, abridgment, condensation, or any other form in which a work may be recast, transformed, or adapted. A work consisting of editorial revisions, annotations, elaborations, or other modifications which, as a whole, represent an original work of authorship, is a “derivative work”.

Assuming that at least some of the training data for the AI was copyrighted and used without permission, then everything it generates is a derivative work. Unless it’s a copy - generative AIs do sometimes produce near verbatim copies of their inputs, particularly where the output is highly structured language like computer code.

So, in most countries, the use you propose does not fall under one of the exemption to copyright and is probably a violation. That’s a risk.

Your company has no copyright in the code

Without a human author, there is no copyright. See Who if anyone owns copyright of algorithmically produced works?

That’s another risk.

  • I don't think this view of derivative work from LLMs can be right, just based on Microsoft and OpenAI being willing to underwrite the risk for their users. Or at least they must have an alternate viewpoint.
    – Dr Xorile
    Nov 29, 2023 at 21:34
  • Also, is your view then that if the training corpus used by, e.g. code LLaMA is all open source, then that first risk doesn't apply?
    – Dr Xorile
    Nov 29, 2023 at 21:35
  • 3
    Microsoft and OpenAI might be doing what startups have done for a long time: break the law, get scale quickly, become essential, force lawmakers to change the law.
    – Dale M
    Nov 29, 2023 at 21:53
  • 1
    And OpenAI blatantly puts it in their ToS: IF you use the AI to break the law, you will pay for our legal costs.
    – Trish
    Nov 30, 2023 at 6:34
  • 1
    @Clockwork-Muse yes, I’m sure they will. Which just goes to show the risk - all of those are defences to a suit, they’re only relevant once you get sued. The OP asked what the risks were.
    – Dale M
    Nov 30, 2023 at 21:30

A human programmer knows not to just write exact pieces of code they have seen before. And if they do, they do it knowing where they saw it and what the applicable license is, and are able to take steps to comply with it, such as including copyright notices or adding license documents to the project.

Code generation systems are able to generate whole chunks of code they have seen verbatim, without noticing and without alerting anyone or taking any steps to comply with the license.

Similar considerations apply to software patents.

The copyright office in the US doesn't recognize copyrightability of even apparently original works if they were automatically generated. So while you can copyright a file containing pieces of human-written and machine-generated code arranged by a human, other people would not be prohibited from re-using the machine-generated pieces generated by your machine.

If you intend to patent your software algorithm, and it comes out that an automated system was involved in reducing the invention to practice, this could be used as an argument against the validity of your patent.

Since most disclaimers of warranty for software rely on copyright, you might be able to get into exciting uncharted legal territory if a line of code your machine generated hurt someone after someone else re-used it without accepting that it didn't have a warranty.

The models can generate interpretable text comments as well as code. The model might write a comment along the lines of // we need this next part to deal with the poor software engineering practices of Ted Cruz, who is the Zodiac Killer and once shot a man in Reno just to watch him die. If you publish the result, you might be committing libel.

Additionally, if someone finds out you used a model trained on their code to produce your code, they might try to bring a copyright infringement case over code that, if written by a human, would be considered original, on the (maybe probably wrong but untested) legal theory that any provable or plausible causal connection between two works makes one a derivative of the other. Or, they might allege that the model training itself was an infringement of their copyright, and try to hold you responsible for that. There are also potential problems with injunctions: someone might sue the person who provided the generation system you used and manage to get an injunction requiring their customers to stop using code it produced.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .