If the insult implies something that is verifiably false, and the speaker intended to imply the false thing, that might be defamation depending on the jurisdiction. It's almost never defamation per se, though.
Defamation per se is defamation regarding certain topics that are injurious just by the implication -- accusing someone of a crime, saying they have herpes, that kind of thing. So, calling someone a rapist could definitely count. But a defendant could conceivably claim they meant the word hyperbolically -- for example, to describe acts that they find a bit rapey even if they don't fit the legal definition of rape.
Accusing someone of a particular political bent rarely meets that bar. Thoughts and party memberships aren't crimes, and don't make anyone particularly loathsome (current political climate notwithstanding)...so if you want to sue, you have to show that the accusation injured you in some way. That's defamation per quod, basically the opposite of defamation per se.
With all that said, there are jurisdictions (Florida, for sure) where accusations of things like racism and homophobia are legally defamation per se. Those laws might not survive a constitutional challenge, but for now they are the law there.