Let's clarify the point: the material is most likely illegal per se
In the laws of many jurisdictions, the definition of child pornography extends from depicting sexual acts with a minor to depicting sexual acts with an apparent minor.
As an extreme example: in Australia, drawn sexual comics with Anime aesthetics are banned wholesale, and England and Wales similarly made photorealistic and even more abstract styles of such material illegal. AI art would fall somewhere under those.
In other countries, like the US or Japan, legality is based on the material not being obscene. Obscenity often is a somewhat subjective test, like in the US the standard for a time was "I know it when I see it" (Jacobellis v. Ohio, 378 U.S. 184 (1964)), but was refined into the Miller Test in 1973:
- Obscene material is not protected by the First Amendment. Roth v. United States, 354 U.S. 476 , reaffirmed. A work may be subject to state regulation where that work, taken as a whole, appeals to the prurient interest in sex; portrays, in a patently offensive way, sexual conduct specifically defined by the applicable state law; and, taken as a whole, does not have serious literary, artistic, political, or scientific value. Pp. 23-24.
- The basic guidelines for the trier of fact must be: (a) whether "the average person, applying contemporary community standards" would find that the work, taken as a whole, appeals to the prurient interest, Roth, supra, at 489, (b) whether the work depicts or describes, in a patently offensive way, sexual conduct specifically defined by the applicable state law, and (c) whether the work, taken as a whole, lacks serious literary, artistic, political, or scientific value. If a state obscenity law is thus limited, First Amendment values are adequately protected by ultimate independent appellate review of constitutional claims when necessary. Pp. 24-25.
- The test of "utterly without redeeming social value" articulated in Memoirs, supra, is rejected as a constitutional standard. Pp. 24-25.
- The jury may measure the essentially factual issues of prurient appeal and patent offensiveness by the standard that prevails in the forum community, and need not employ a "national standard." Pp. 30-34.
In a totally different definition, japan has its Penal Code of Japan Section 175 reads (in translation) much more like a subjective test, which is often called out to demand conformity (and censor bars):
Chapter XXII, Article 175. Distribution of Obscene Objects: A person who distributes, sells or displays in public an obscene document, drawing or other objects shall be punished by imprisonment with work for not more than 2 years, a fine of not more than 2,500,000 yen or a petty fine. The same shall apply to a person who possess the same
for the purpose of the sale.
Who is in possession of illegal material?
This depends:
- IF the AI was trained on illegal material, the makers of the AI had in their possession material of which possession is illegal.
- IF the hosting company stores the material generated by the user,
and it is of the banned type once evaluated, then the company where the image is hosted might be in possession.
- In the US, due to Section 230 (of Title 47), the hosting company however is blameless for user-generated and user-stored content. The User is fully to blame.
- The User, by properly downloading the file actively gets into possession of illegal material, which would make them liable for that