Someone is yanking your chain. The law regarding birth certificates §31-10-9 states that "A certificate of birth for each live birth which occurs in this state shall be filed with the State Office of Vital Records within five days after such birth and filed in accordance with this Code section and regulations of the department" ("shall" means it is mandatory). The law regarding issuing certified copies of vital records, §31-10-26, says that
(a) In accordance with Code Section 31-10-25 and the regulations
adopted pursuant thereto: (1) The state registrar or local custodian,
upon receipt of a written application, shall issue: (A) A certified
copy of a vital record in that registrar's or custodian's custody or
abstract thereof to any applicant having a direct and tangible
interest in the vital record
(a parent has an obvious direct and tangible interest in their child's birth certificate). There is no legal basis for denying a person the right to obtain a birth certificate, and a law prohibiting issuance of a birth certificate to a person who owed money to a hospital would be contrary to the Equal Protection clause of the Constitution. However, the hospital is not obligated to give copies of birth certificates, and probably cannot issue a certified copy ("(d) No person shall prepare or issue any certificate which purports to be an original, certified copy or duplicate of a vital record except as authorized in this chapter or regulations adopted under this chapter"). USCIS states that "The only birth certificate acceptable for Form I-9 purposes must be an original or certified copy of a birth certificate issued by a state, county, municipal authority or outlying possession of the United States, bearing an official seal (hospitals register births, on which basis government registrars certify births). Your home country obviously cannot issue a birth certificate: only Georgia can (and must).