No. Arousal does not constitute duress.
But there are a few other and related legal arguments that could come up as defenses to the enforcement of an agreement in this context.
One defense is that there is no consideration for the agreement, if it amounts to a promise to make a gift in the future. Promises not supported by consideration (getting something for giving something) are not enforceable as contracts unless the person not giving consideration nonetheless reasonably acted in reliance on the promise, in which case it can be enforced under the equitable doctrine of promissory estoppel, rather than under the usual law governing breaches of contract.
A second defense is that the agreement is supported only (or at all) by sexual services as consideration, i.e. that it is "meretricious" and hence, void as contrary to public policy since prostitution is illegal. The exact requirements to invalidate a contract as meretricious vary from jurisdiction to jurisdiction. Merely having a sexual relationship with someone does not invalidate a contract that would otherwise be valid and does not have sex as part of the consideration for the contract.
A third defense is that the agreement was secured through "undue influence" and voidable for that reason. In context of the validity of a contract, "undue influence" means that someone has such a pervasive capacity to overcome the free will of the person alleged to have been unduly influenced that the agreement was not truly a free and voluntary act of that person. Typically, this would involve a person influenced who had something akin to early stage dementia who was marginally competent to enter into a contract but more easily swayed than an ordinary person, or to an abusive and controlling relationship situation in which they were economically dependent. This would not be applicable, for example, in the case of an ordinary working professional who was indulgent towards a lover.
Agreements entered into by people who are in a "confidential relationship" with someone, or a "fiduciary relationship" are presumed to be procured with undue influence until that presumption can be rebutted, and a lover would often, although not always, be something who is in a "confidential relationship" with someone. But in the case of a mentally sound and free person, overcoming that presumption would be fairly easy.
Also, while merely being aroused wouldn't constitute duress, if the lover was able to secure the agreement not due to arousal, but because the lover has threatened to reveal the existence of the relationship to the public, even if the relationship was not illegal and instead merely damages the reputation of the person agreeing, that would be criminal extortion and might also rise to the level of legal duress forming a basis to invalidating the contract.