Strictly speaking, that principle isn't even true everywhere in the US. The maxim "nulla poena sine lege" (i.e. "no punishment without a prior penal statute") was historically applicable to civil law systems, such as are found in continental Europe. In common-law systems, there was never a tradition in which a crime wasn't a crime unless it violated a penal law, because crimes themselves were traditionally defined by court precedent instead of by statute.
In US federal court, the only allowable common-law offense is contempt of court. This is due to a court decision (United States v. Hudson), in which the Supreme Court ruled that federal courts do not have the constitutional authority to hear a case in which someone is accused of committing a common-law crime. Even so, and even though there is a federal contempt statute, the Supreme Court has ruled that contempt is an inherent power of any court, and statutes around it only regulate the power (but the power would be there even without a statute).
At the state level, some states have explicitly passed laws saying something is not a crime if it doesn't violate the penal code (although this doesn't necessarily apply to contempt); see section 6 of the California Penal Code for an example. In other states, like Florida, common-law crimes still exist; Florida has a statute saying that any common-law offense is still a crime unless a statute has explicitly covered that same subject matter (section 775.01), and specifies a generic penalty for anything which is an offense at common law and not addressed by any Florida penal statute (section 775.02). While this is sort of statutory (as it's a statute giving the penal provision), it's also basically not (as no statute has to say "X is illegal," because it's enough that English common law makes X illegal).