Another answer is correct, but doesn't get to the point as quickly and directly as this one does.
Are there in fact felony charges that could be brought in the USA on
the basis of a "fraudulent marriage" entered into only to avoid
deportation?
Yes. As the other answer points out, technically, it wouldn't come up in these exact circumstances in the last few decades. But yes, this is potentially an issue in almost all marriage based visa cases. Very similar cases where the non-citizen spouse is not under removal or deportation proceedings but is at risk of being in them, are common.
If so, what factors would make the marriage fraudulent?
If the marriage is really a sham and for immigration purposes only.
Definition of Marriage Fraud in U.S. Immigration Law
A sham marriage is one that is entered into in order to get around
("evade") U.S. immigration laws. (See I.N.A. Section 204(c).)
For a marriage to be valid under U.S. law, it is not enough that the
couple had a real marriage ceremony and got all the right governmental
stamps on their marriage certificate. They have to intend to live in a
real marital relationship, namely to establish a life together,
following the marriage ceremony—and they must prove their intention
through their actions. If the couple doesn't intend to establish a
life together, their marriage is a sham.
Another way in which an immigration application based on marriage will
be considered fraudulent is if it isn't legally valid. Say, for
example, that you are already married to another person, and were
never able to get a legal divorce. Even if you truly love your new
spouse, this current marriage is invalid. Applying for U.S. lawful
permanent residence (a green card) on the basis of an invalid marriage
is, indeed, considered fraudulent. Even if you get away with it in the
short term, your green card and eventual U.S. citizenship could be
taken away on the basis of this fraud.
What the U.S. Government Expects a Real Marriage to Look Like
What is
the U.S. government's view of a typical marriage? The statutes and
regulations don't go into detail on this, so the following comes from
a combination of court cases and attorneys' experiences. You certainly
are not required to match every description on this list—but the fewer
you match, the greater the likelihood that you will face lots of
questions and close scrutiny of your application.
The "normal" married couple has a fair amount in common. They share a
language and religion. They live together and do things together, like
take vacations, celebrate important events, birthdays, and holidays,
join clubs or gyms, and have sex and children.
Typical couples also combine financial and other aspects of their
lives after marriage. They demonstrate their trust in one another by
sharing bank and credit card accounts and ownership of property, such
as cars and houses. They celebrate each others' birthdays and meet
each others' families.
And they might even have arguments and marital problems. Ideally, they
will visit a counselor or other trusted adviser to help work these out
(and these visits themselves can become evidence of a valid marriage).
High Level of U.S. Government Suspicion Regarding Marriage Fraud
Detecting marriage frauds is a top priority for U.S. Citizenship and
Immigration Services (USCIS) and its affiliated agency, Immigration
and Customs Enforcement (ICE). Based on a decades-old survey,
government officers still sometimes say that up to 30% of marriages
between aliens and U.S. citizens are suspect. That statistic has been
shown to be deeply flawed, but its legacy lives on.
(Source)
It goes mostly to whether the couple live their lives as a married couple.
One immigration lawyer online estimated that as many as 10% of marriage visas in the U.S. involve marriage fraud, at the high end of the range of what is a reasonable estimate.
Teaching people what they need to do to avoid having marriage visa applications denied, to avoid having marriage visa applications revoked, and to avoid having citizenship applications of people with marriage visas denied, is a whole cottage industry in the United States.
For more background a law review article on the subject of immigration marriage fraud in the U.S. can be found here.
If there are such laws, are they really enforced?
Yes. Frequently. In 2005, for example, 8% of marriage visa applications were denied by U.S. immigration officials for marriage fraud. See, e.g., a 2019 press release about a marriage fraud prosecution, and another 2019 press release about another marriage fraud prosecution.
Or is it something like the laws against adultery in the seventeen
states which still have them (where breaking the rules is socially
frowned on but usually not prosecuted criminally)?
It is not. Adultery is neither a requirement for a sham marriage, nor sufficient grounds, standing alone, for a sham marriage.