It isn't 100% clear from the question if a case has been filed in court, or someone was just planning on filing a lawsuit, which is an important fact. It seems like the ex filed a court case and you hired attorneys who responded.
If there is a court case filed, that can't just be abandoned until all the i's are dotted and t's are crossed in the eyes of the court. The lawyers can't quit unless the court gives them permission to do so.
Usually, lawyers are entitled to be paid for all of the work they do and out of pocket charges they incur in a case, until it is wrapped up, even if some wrap up work happens after the event that determines the final outcome of the case like your ex deciding to abandon his arguments.
But, otherwise, if there isn't a pending court case, you normally have the power to tell your lawyers to stop everything and give you the moment left (if any) in your retainer. At first read, it almost sounded as if your lawyers are willing to do that, but are warning you that your ex might continue to be a problem after the lawyers quit and that if that happens, it will be more costly and time consuming to start all over dealing with the threatened lawsuit that your ex made, than it would be to get it over and done with now. But, upon closer inspection, it seems that there is a pending lawsuit and that this is the issue.