only federal law applies to foreigners
This is not correct. For example, there is no federal law criminalizing murder generally. (There are federal laws criminalizing murder in certain specific circumstances.) If you killed the shopkeeper around the corner, you probably would not have violated any federal law. You will of course be subject to prosecution, conviction, and punishment under California state law.
So I am wondering, what laws do apply to me?
US federal law, California law, and the ordinances of the municipalities you live in or visit all apply to you. There may also be laws from your country of citizenship that apply to you even though you are not presently in its territory.
Do I have to comply with State laws?
Yes.
What if State and Federal laws contradict each other?
It depends on the nature of the conflict. If one set of laws forbids something and the other does not, then the thing is generally forbidden. The situation is weird with respect to marijuana, however, and I am afraid that I have no idea how it applies to you as a foreign nonimmigrant resident of California.
If Federal and state laws conflict more directly, then federal law applies pursuant to the supremacy clause of the US constitution:
This Constitution, and the Laws of the United States which shall be made in Pursuance thereof; and all Treaties made, or which shall be made, under the Authority of the United States, shall be the supreme Law of the Land; and the Judges in every State shall be bound thereby, any Thing in the Constitution or Laws of any State to the Contrary notwithstanding.
As ably pointed out in user6726's answer, the truth behind what you heard probably lies in immigration law. That is, it's not that California law doesn't apply to foreigners, but that certain federal laws don't apply to US citizens. A US citizen cannot be removed or excluded from the US under immigration law, but you can be.
December, 2021:
Revisiting this question and its answers, it occurs to me that it might be worth mentioning that treason can only be committed by US nationals (including US citizens). That's a law that doesn't apply to foreigners, though, not one that does. Still, any act that would constitute treason if committed by a US national would surely constitute one or more crimes for which a foreigner could be convicted, so not being liable for treason isn't much of an advantage.