If a used car buyer backs out of buying a car do I have to return the
deposit even if i didn't explicitly say it is “nonrefundable”?
Yes, that is very likely. With the little information you provide, it seems that the desisting buyer would be entitled to the refund for the very reason he stated.
One of the cornerstones of contract law is that a contract or agreement be entered willfully and knowingly. The latter item appears to be missing here, at least with respect to the desisting buyer. It is very possible that the desisting buyer would not have made the deposit had he known that it was non-refundable.
It is noteworthy that your question does not really contain sufficient detail of the conditions of the agreement (preferably the exact wording) and who formulated them. Thus, it is impossible to ascertain the parties' intent and hence each one's rights.
Your statement that "I never explicitly said it was nonrefundable" suggests that the party who formulated the condition(s) is you. If that is the case, you created an ambiguity which now entitles the buyer to apply the doctrine of contra proferentem, and therewith to the reimbursement.
That being said, you might be entitled to keep the money if you are able to prove that:
- the policy of nonrefundability of
deposits is so commonplace --in similar contexts-- that there was no
need to make it explicit in your matter; or
- during three-day period you declined other bids which
would have been more profitable to you and now those bidders are no longer interested, whence principles of
equity entitle you to the minimum between the deposit and the irreversibly lost income (see also the Restatement
(Second) of Contracts at § 272 and 377).