Broadly speaking, a writ of mandamus is a court order to a government official to do its job properly, typically by performing some nondiscretionary task.
It is commonly sought against officials in the executive branch, where there is not always a clear process for appealing a decision to take or not take some action. In the judiciary, there's nearly always the possibility of an appeal, so mandamus is not particularly commonly granted against a judge.
There is, however, plenty of precedent for writs of mandamus against courts. One example that quickly comes to mind is Richmond Newspapers, Inc. v. Virginia, 448 U.S. 555 (1980). In that case, a trial court barred the press from a murder trial. A newspaper challenged the order closing the courtroom, but the judge refused to let reporters in. The paper sought a writ of mandamus ordering the court to permit coverage of the case, and the U.S. Supreme Court granted the writ.