I don't understand this concept called "spousal privilege". Regardless of whether a defendant is a wife or husband in relation to a potential witness, the latter can always refuse to say jack or just go with the "I can't recall" thing. What, will they torture them? What's the point of these "privileges" then?
Regardless of whether a defendant is a wife or husband in relation to a potential witness, the latter can always refuse to say jack or just go with the "I can't recall" thing. What, will they torture them? What's the point of these "privileges" then?
There are actually two separate spousal privileges, the confidential communications privilege and the testimonial privilege.
Confidential communications with a spouse are not admissible in evidence, even if the spouse wants to testify. This privilege is like an attorney-client privilege, or a psychologist-patient privilege, or the privilege that attaches to confessions made to a priest, reflects the long standing assumption that communications made in reliance on a confidential relationship should not be betrayed in court. There are often exceptions to the spousal privilege for domestic violence and child abuse. It applies even if the couple that was married at the time that the communication was made is no longer married.
Part of the theory of the confidential communications privilege with a spouse is that these communications wouldn't have happened in the first place but for the privilege, and that communications between spouses serve a socially desirable end overall that should be encouraged, by strengthening marriages.
The testimonial privilege is the privilege against testifying against a current spouse. In some jurisdictions this belongs to the testifying spouse, and in some jurisdictions it belongs to the spouse testified against. In Florida, it belongs to the testifying spouse in federal court and is not available in state court at all. This applies only to a current spouse and is designed to avoid forcing someone to have to choose between obeying the law and loyalty to their spouse. This usually has more exceptions than the confidential communications privilege and some jurisdictions don't have it at all.
The law presumes that people who testify under oath will tell the truth and retains the right to punish them for contempt of court or perjury if they do not. The fact that someone can lie on the stand in ways that sometimes can't be caught is besides the point of whether it is fair to force a spouse to do that. Historically one goal has been to prevent domestic violence against a testifying spouse that might occur if the testifying spouse testifies truthfully but negatively against their non-testifying spouse. Also, there is a sense that the testimony offered may have more to do with the health of the marriage (being more or less favorable based upon that without regard to whether the testimony is truthful) and hence isn't reliable.
It also prevent discovery in the course of a lawsuit from one's spouse, which could otherwise be used as a dirty litigation tactic to impact a litigant's family in a civil case primarily for purposes of harassment.
Contempt of court sanctions against non-cooperative witnesses are not uncommon. Lots of witnesses aren't very sophisticated and aren't good at paying attention. When you start answering "I don't know" regarding your address, when the last time you saw your spouse in person, what kind of car you have, and whether you know what a copying machine is, it is often pretty easy to spot an intentionally uncooperative witness. Often there are prior unsworn statements or the witness is crystal clear on some points and yet claims no recollection on others.
Procedurally, if the judge is convinced that someone is trying to obstruct the court a judge makes a finding of fact to that effect based upon observations that could support that conclusion in the testimony and evidence, higher courts on appeal have to defer to those findings. Appellate courts can't second guess trial court credibility determinations. So, it is a lot easier to establish grounds for contempt of court than one might theoretically expect.
Perjury prosecutions are exceedingly rare (on the order of 1 per 200,000 people per year), and a majority of those prosecutions do not involve court testimony. But the legal prospect of being convicted of a felony for lying under oath in nonetheless a powerful one because a lot of tactical and practical decisions in court are based upon worst case scenarios, and it does happen now and then, a few times a year at least, in most states.
When you are giving evidence on the stand you cannot just
refuse to say jack or just go with the "I can't recall" thing"
Refusing to tell what you know would be contempt of court. Lying would be perjury.
Spousal privilege simply frees the spouse from having to either commit one of those or testify against their spouse.
Ohwilleke's answer provides the basic modern rundown of the theoretical basis for spousal privilege (or privileges, rather, as his answer also details). But we can also jump into the way-back machine to determine where that came from.
We'll start with:
Protection against self-incrimination
Common law jurisdictions trace their origins to the common law of England; the US legal system is a common law jurisdiction, naturally inheriting English legal sensibilities from the English colonial settlers. This system arose over hundreds of years, tracing a complicated evolution (and at times, arguably, devolution). One of the later developments, around the 15th century, was a reaction to the perceived abuses of what was, at the time, the dominant form of justice in continental Europe, and which had seeped into the UK via the Star Chambers: the inquisitorial system. This methodology was (in)famously deployed by the Catholic Church, including the Spanish Inquisition among others. It has likely been used for much of the history of human civilization, and in fact the methods of the police in the US can be reasonably described as inquisitorial.
In the inquisitorial system, as it was often used around the 15th century, the authorities figure out what they think the truth is, then they call you in to testify about it, and if you don't testify to what they expect you to say they punish you. This was easily abused: decide ahead of time that X is guilty, regardless of the actual truth of the matter, and X gets punished one way or another. The penalties for not saying what they wanted you to say were made to be so harsh (torture) that it was highly unlikely the confused wouldn't confess to anything and everything ordered of them just to get it to stop. And an outright confession is very powerful in the minds of the people. You really couldn't get any better than an admission straight from the horse's mouth to convince the masses that justice had been done.
Many common law protections, such as the right to not incriminate yourself in criminal matters, were cemented to prevent such abuses. While a right against self-incrimination has a history long predating the Star Chambers of the 15th century, the Inquisitions, etc., as well as a history of it being subverted along the way, it was about the time of the Star Chambers that proponents of the right really amped up the rhetoric and helped solidify the protection (though this took some time). With such a right in place, you couldn't force someone to admit guilt, you had to go about and prove it, to a jury of the accused's peers (not the accused's accusers and investigators), with evidence and witnesses, which could be countered by evidence and witnesses provided in defense of the accused (plus cross examinations).
So how's that tie to the topic at hand?
Marriage as Union
You may often hear marriage get talked about as a (holy) union of two people. What you might not realize is that this used to be taken very seriously at the legal level. For a long time in common law jurisdictions, married people were treated as one legal entity. They were not two people. They were one, single person/entity before the law.
Thus if one spouse did not want the other to divulge something that could be used against them, they were granted the right to impose this: the spouse, in this legal fiction, was them, so their express desire to not have it said is the same thing as not wanting to say it themselves. With a right against self-incrimination, since your spouse is you as far as the law is concerned, neither can your spouse be forced to testify against you. Just as your internal thoughts cannot be forcibly laid bare to the court, nor can your conversations with your spouse be laid out. Within this legal fiction, these are one and the same thing.
Mind you, the full history of this, too, is rather more complicated than I just made it out. The Catholic Church itself (and other religions) has long been a proponent of such spousal privileges, and the belief that a married couple is one entity before God. After all, they advocate that marriage is a sacrosanct, holy, and inviolable union as part of their religion. Attempting to exploit that to gain leverage in proceedings would then rather inevitably be a gross sin.
In the modern era, including in the US, this legal fiction has largely dissipated, with any of its remaining legal consequences provided other rationales, such as in Ohwilleke's answer.