That release needs to be provided in writing to each legal entity (read: office) that provides you with medical care. Providing the release verbally (ie: 'I say that my assistant speaks for me') does not cut it andit; the offices that agreed are technically in jeopardy of some hefty fines and are just trusting you won't turn around and report them. Most offices are going to have their own forms for this and won't accept some generic letter that you write up, sign, and send all of them.
Have your assistant call each office and ask for them to fax over a copy of their release form. List your assistant as an authorized contact, sign it, and send it back to them.