For state courts, it depends on the state. Pro hac vice appearances like this are probably most common in New York, in relation to international commercial disputes. It is also common for foreign attorneys to take the New York bar exam - several thousand people do that each year - in which case they have a springboard to appear pro hac vice in other states. The NY exam is well-understood in this way, in terms of training and support that might be offered in the country of origin. But one-off appearances, particularly by in-house counsel, also take place routinely.
Many states, including California, also have a category of "foreign legal consultant", someone who is retained to advise on the law of the specific foreign jurisdiction in question, but who isn't allowed to appear in court.
In the current version of the California Rules of Court, pro hac vice is covered in Rule 9.40. On its face, this rule only applies to attorneys who are "eligible to practice before the bar of any United States court or the highest court in any state, territory, or insular possession of the United States". There is a broad exception in 9.40(h),
This rule does not preclude the Supreme Court or a Court of Appeal from permitting argument in a particular case from a person who is not a licensee of the State Bar, but who is licensed to practice in another jurisdiction and who possesses special expertise in the particular field affected by the proceeding.
"Another jurisdiction" is not facially limited to U.S jurisdictions, so it may be that the higher state courts could admit a foreign attorney. I do not know of an example either way.
In general, California's licensing requirements for lawyers have often been strictly interpreted. The State Bar Act (Business and Professions Code, 6000-6243) says that:
No person shall practice law in California unless the person is an active licensee of the State Bar.
An attorney appearing pro hac vice is temporarily registered with the State Bar once their application has been approved by the court. This general licensing regime, which aims to protect the public against dishonest or incompetent practitioners, has been held (e.g. in Birbrower v. Superior Court 17 Cal. 4th 119 (1998)) to imply
a strong public policy favoring the practice of law in California by licensed State Bar members.
Occasional admission pro hac vice does not extend to doing regular legal work of other kinds, including giving legal advice, preparing documents, and so on - and certainly not to maintaining a permanent physical office in California, or advertising to the public. If you would like to do those things then the state would like you to take their own bar exam, in which case (if you pass!) you are in the same situation as any other California lawyer.
There is a fairly recent exception for international commercial arbitration (not applicable to normal arbitration about things like disputes over personal property). The Code of Civil Procedure, 1297.185, now says that if you are a
member of a recognized legal profession in a foreign jurisdiction, the members of which are admitted or otherwise authorized to practice as attorneys or counselors at law or the equivalent,
and meet various other conditions, then you can participate in arbitration work on a "fly in, fly out" (FIFO) basis. You don't need to register or pay a fee. (But, per 1297.187, you still need pro hac vice permission to appear as counsel in an actual court.) Additionally, if you FIFO, the Federal government will have visa requirements at the point you try to FI.