The GDPR will probably not allow a successful demand for erasure in these circumstances. The company does not need to argue "that they can't delete user information for fear of be liable for.something" because GDPR art 17 1(c), 3(b), and 3(e) already make that case for the company.
Lets look at the relevant law.
GDPR Article 17 reads in relevant part:
(1) The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay and the controller shall have the obligation to erase personal data without undue delay where one of the following grounds applies:
(1) (a) the personal data are no longer necessary in relation to the purposes for which they were collected or otherwise processed;
(1) (b) the data subject withdraws consent on which the processing is based according to point (a) of Article 6(1), or point (a) of Article 9(2), and where there is no other legal ground for the processing;
(1) (c) the data subject objects to the processing pursuant to Article 21(1) and there are no overriding legitimate grounds for the processing, or the data subject objects to the processing pursuant to Article 21(2);
...
(3) Paragraphs 1 and 2 shall not apply to the extent that processing is necessary:
(3) (a) for exercising the right of freedom of expression and information;
(3) (b) for compliance with a legal obligation which requires processing by Union or Member State law to which the controller is subject or for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller;
...
(3) (e) for the establishment, exercise or defence of legal claims.
Art 17 Par 1 (a) only allows erasure when data are
"no longer necessary". But depending on how the purposes were worded by the Controller, they may well still be necessary
Art 17 Par 1 (b) only allows erasure when consent is withdrawn and consent was the legal basis for processing. But a contract with the user is a much more likely basis for processing for a financial service.
Art 17 Par 1 (c) requires erasure if the user objects and there is no "overriding legitimate grounds for the processing" but the needs to maintain financial records of a transaction is probably an "overriding legitimate ground".
In any case par 3 alloys erasure to be refused "for compliance with a legal obligation" or "for the establishment, exercise or defence of legal claims". Financial records may well fall into one or both of these categories, at least as regards transaction data.
In short it is far from clear that the GDPR will require erasure in these circumstances.