Technically, they were born in the United States, at least under US law, and the law of any existing nation. The United States never recognised the Confederacy, nor did any other country. There are occasional wingnut groups who declare themselves separate from the United States, but a simple declaration of secession does not create a country – you also have to win the war.
The Supreme Court affirmed that the secessionist states were always an integral part of the Union in Texas v. White, 74 U.S. (7 Wall.) 700 (1869). Since they only had jurisdiction if Texas was a state at the relevant time (some time on or before January 15, 1865), by accepting the case, they implicitly affirmed that Texas was part of the Union. Further, in the judgement itself, they explicitly stated that the acts of secession of not just Texas, but every rebel state were “absolutely null”.