A judge has the authority to determine what law applies to a case and to instruct the jury accordingly, and also has the authority to determine which evidence is admissible.
Presentation of a defense, in practice, involves presentation of evidence supporting a legal theory.
In order to be admissible in evidence in an evidentiary hearing or trial, the evidence must be relevant to a legal theory that is in some way connected to the evidence. If no reasonable juror could make a ruling establishing that a legally recognized defense was established based upon the proposed evidence (especially if the proposed evidence is prejudicial to the prosecution case on the basis of reasoning that is not a legally valid defense) it can be excluded.
For example, evidence in support of the theory that the defendant murdered the victim because the murder victim raped the defendant's sister six years ago, might very well sway a jury to acquit the defendant. So a defense attorney might want to make this argument. But, this is not a legally recognized justification for murder, so evidence in support of this defense would be excluded as irrelevant by the judge.
In federal court, and in states with rules of evidence based upon the federal rules of evidence, the primary legal authority behind this is Rule of Evidence 402:
Rule 402. General Admissibility of Relevant Evidence
Relevant evidence is admissible unless any of the following provides
the United States Constitution; a federal statute; these rules; or
other rules prescribed by the Supreme Court.
Irrelevant evidence is not admissible.
Some states also have procedural notice requirements for certain kinds of affirmative defenses.
For example, if someone is arguing an alibi, a notice of an intent to present this defense must be provided by the defense a certain number of days before trial, so that the prosecution can develop the very different in kind type of evidence needed to rebut that defense, rather than having someone acquitted due to surprise when rebuttal evidence exists but the prosecution doesn't know in advance to locate the necessary witnesses and evidence to rebut this kind of defense.