Do judges have the discretion to apply older laws or rulings, or do they have to follow the new legislation regardless?
By historical precedents, I am referring to the common law that has developed over time through various judicial rulings.
The new law wins. That's rather the point of passing a new law-- the legislature wants to change the current law of the land whether that is based on statutes or court rulings.
Of course, there are caveats. Prior judicial rulings may have relied on an interpretation of a source of law that supersedes the legislature in question (i.e. based on an interpretation of the state or federal Constitution or based on a federal law that supersedes the state law). In that case the new law would be found unconstitutional or ignored. Or the new law might have an ambiguous interaction with current law that courts would have to resolve. The new law might clearly make X illegal but there may be legitimate questions about whether it intended to make previously legal action Y illegal as well. It is, after all, very difficult to write a law that covers every possible fact pattern one would encounter in the real world.
I agree with the answer from @JustinCave and I am just adding a concrete example.
In Colorado, the state supreme court decided some cases that interpreted the common law of premises liability (i.e. the liability of a landowner for injuries that take place on their land) in the state. The legislature didn't like the legal rules that resulted from those decisions.
In response, the legislature passed Colorado Revised Statutes § 13-21-115 Actions against Landowners, often referred to as the Colorado Premises Liability Act, that overruled the state supreme court holdings and adopted a different set of rules for premises liability.
(Actually, it was a bit more complicated than that, with at least two rounds of court decisions and two rounds of legislative acts overruling those court decisions.)
In countries with common-law systems, like the United States a new statute that contradicts a previous common-law precedent. A judge does not have authority to disregard the new statute in favor of the older precedent.
However, a new US state statute cannot override a previous federal law, a federal constitutional provision (or a court ruling interpreting such a provision), nor a state constitutional provision. A new state statute which contradicts any of those will be ignored, or held top be unenforceable, at least to the extent of the contradiction. Similarly, a new local ordinance cannot override an existing state law, unless the state law allows for this.
If a new state statute modifies some older common-law rules, but does not directly contradict others that apply to a situation, a court may well apply as many of the previous rules as were not directly changed by the new law. If it is not clear whether the new law was intended to change an existing precedent or not, a court may well prefer to assume that it does not, and rule so as to apply both the new law and the old rule, if that is possible.
Note that the rule for an older existing state statute is the same as for a previous common-law precedent. A new law overrides any prior contradictory law (at the same level) if it is clearly intended to do so. But when the new law does not clearly repeal or modify the old one, a court may attempt to apply both.
A comment asks:
So common laws are more like subordinate "guidelines" to "real laws" from actual legislation?
No, common-law rules are laws and have the same status as statutes, but the rule is always that at the same level, the newer law overrides any older law, and the statute is treated as newer than any common law.
I should add that I agree with the answers by Justin Cave and ohwilleke, and I have upvoted both. But I wanted to make the point tht "newer law overrides older law at the same level" is a general rule, and not something that applies only to common-law rulings.