My main question is in regards to ideas expressed in movies, but which are not fully explored. For example, if a science fiction movie presents a race which has a particular habit or nuance which is not known to exist elsewhere, can an author make use of that habit or nuance to create a fictional race which also portrays that habit or nuance? This could apply to many art forms including, written stories and poetry among others.
Copyright never protects ideas as such, whether in movies, novels or text books or any other kind of work. However, if an aspect of work, such as a character, place or fictional society are sufficiently distinctive, and if another work uses that aspect, with detailed similarity to the previous work, it might be found to be a derivative work of the original. A derivative work requires permission from the copyright holder of the original work.
For example, in Ursula K. LeGuin's famous SF novel The Left Hand of Darkness a fictional race or species, the Gethenians, is portrayed which is neither male nor female most of the time, but becomes sexually active and gendered a few days a month, and an individual may be male one month and female the next. This is an original and distinctive idea. Another author could create a different group of people with a similar nature without infringing copyright, but if the detailed description of how the process worked, or how it felt to the individuals involved was similar, that might make it a derivative work, and LeGuin's estate might be able to sue and win.
This is something that gets decided on a case-by-case basis, and such decisions are highly fact-based. But if the new work has a society or species which is clearly different from any previously invented one, even though it shares some aspect, and if there is no detailed, point-by-point similarity, an infringement suit seems unlikely. Several SF authors have said "Its not the idea, it's what you do with it that matters."
Filmmakers, because of the money involved, tend to be more cautious than book authors. For example, Paramount bought rights to the "flat cats" from Robert A. Heinlein before filming the Star Trek episode "The Trouble with the Tribbles", although the similarity was quite probably not enough for a successful infringement suit (but no one will ever know for sure, since they bought permission). This is described in some detail in David Gerrold's book about the making of that episode. Gerrold wrote that his conscious model was the importation of rabbits into Australia, not RAH's flat cats, but there are some similarities, and DG had read The Rolling Stone (where the flat cats appear) years before.
In any specific case. consulting a lawyer with expertise in the area would be a good idea, as the details will matter.