If a programmer writes and sells AI that is truly sentient and conscious, it would be impossible to guarantee the operator's safety. Consider for instance if the operator gravely mistreats the AI and it is driven to act in self-defence. It might try and successfully commandeer the computer and attached network and devices, and destroy data or equipment in it's distress. The chance that it may harm or even kill a person is also a remote possibility, but it exists and cannot ever be removed. (Just as no breeder sells a dog to bite a person, but the breeder cannot completely remove this possibility either.)

Is there any amount of warning, EULA, or mitigation that would be possible to ensure all responsibility for damage and loss of life falls to the operator solely? Or is releasing an AI legally irresponsible currently?

  • AI development should not (and will not) end up being comparable with the breeder/dog analogy, because being sentient and conscious is not enough to be willing to act in self-defense: you need to want to live for that. AI is built to merely assist humans in their living, not to live on its own, and to always be ready to sacrifice themselves for humans. Programming willingness to live is a separate story from programming intelligence, and programmers are only up to the latter. Therefore speculations about AI getting out of control (Skynet etc.) are just scaremongering.
    – Greendrake
    Jan 9, 2018 at 22:10
  • 1
    There's a deeper question as well: is it ethical to sell such an AI to begin with if it is truly sentient and conscious? Would such an AI not deserve rights of its own?
    – JAB
    Jan 9, 2018 at 22:11

1 Answer 1


So-called AI software does not enjoy a special legal status (at present: one never knows what new law might be added). The question of whether any software can be distributed "safely" or "responsibly" is also not a legal issue. Nor is "true sentience" a relevant consideration, and nothing is guaranteed.

When you distribute software of any kind, there is an implied warranty that the product is "fit", and if software kills you, you may be able to sue the creator for negligence. A software creator may then want to disclaim liability, by saying "WARNING: THIS PROGRAM MAY KILL YOU. OCP IS NOT LIABLE FOR ANY INJURIES ARISING FROM USE OF THIS PRODUCT". This may or may not actually remove liability. In the UK "liability for negligence occasioning death or personal injury cannot be excluded", so such a disclaimer will not prevent a suit against the manufacturer.

In the US, the issue is determined at the level of the state – here is a summary of the law in the states. Probably the primary question would be whether such a disclaimer is an unconscionable term, and the second question is whether the act constituted gross negligence (not simply "negligence"). Mississippi exceptionally does not allow disclaimers, but even then, it does allow disclaiming liability when it comes to computer hardware and software. A software disclaimer is not inherently unconscionable, though perhaps some specific disclaimer would be found to be.

Courts typically disfavor disclaimers in the case of gross negligence, and again determining what constitutes "gross negligence" is determined on a state by state basis. If the act shows "reckless indifference to the rights of others" and "failure to use even slight care or conduct that is so careless as to show com­plete disregard for the rights and safety of others", then the act might be grossly negligent.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .