Is It Legally Binding?
While their customer service sucks, your oral authorization of the charge is legally binding (I take payments that way almost every day in my own business, it isn't an unusual business practice in small professional businesses). You authorize oral authorization of payments over the phone in the credit card agreement that your credit card company sends you every year that you don't read and throw away.
The provider has to collect more information for a credit card payment over the phone than they do for an in person swipe in a credit card machine (e.g. your credit card billing address) and they are fully responsible for wrongful charges if they deal with an imposter. By regularly checking your credit card statements, you can confirm that no incorrect charges are present.
Tax Issues
If you want to take a tax deduction for non-reimbursed medical expenses, you simply need to tote up the amounts your are entitled to from your own records, and put it in the appropriate box on your tax forms. You don't have to attach documentation to your return.
If the IRS disputes your payment, you can offer up your credit card statements and your photograph of the receipts, and if necessary, medical records to show that you received the services, to show that the payment really happened and are deductible. Your credit card company's records, reflected in your monthly credit card statements, are considered very reliable for tax purposes.
You have the burden of proving that the expense was incurred and is of a type that qualifies for a deduction by a preponderance of the evidence in the event that there is a dispute that is litigated, which means that you must show that it is more likely than not that you incurred a deductible expense of that kind in that amount in that tax year.
Privacy Issues
While there are financial information privacy issues associated with this transaction, HIPAA, which covers medical records, normally wouldn't apply to a credit card payment that indicates the person paid, the person paying, the account, the amount and the date, but not a description of the medical services provided or to whom they were provided, which is what is normally on a credit card receipt.
The financial privacy issues are also partially addressed by the provider's merchant agreement with the credit card company which contains terms requiring them to maintain certain kinds of security with respect to your financial information (which is not to say that the provider actually follows all of the requirements of their merchant agreement scrupulously, which is why data breaches happen all the time in businesses both large and small).
