The 14th Amendment says
All persons born or naturalized in the United States, and subject to
the jurisdiction thereof, are citizens of the United States and of the
State wherein they reside. No State shall make or enforce any law
which shall abridge the privileges or immunities of citizens of the
United States; nor shall any State deprive any person of life,
liberty, or property, without due process of law; nor deny to any
person within its jurisdiction the equal protection of the laws.
However, there are many citizens who are or were legally prohibited from voting, including minors and felons (still), women (until the 19th Amendment), and blacks (sporadically, until the 15th Amendment outlawed that). Reynolds v. Sims, 377 U.S. 533 and related cases held that districts within a state have to be comparable in population, but does not directly rule on whether a state can decide to leave a certain geographical area entirely out of the subdivision into districts (the issue there was districts that were lopsided, population-wise). §2 of the 14th Amendment does maintain that
when the right to vote...is denied to any of the male inhabitants ...
or in any way abridged, except for participation in rebellion, or
other crime, the basis of representation therein shall be reduced...
so the state-wide basis for counting representatives would be reduced.
No existing constitutional amendment, federal law, or supreme court ruling expressly denies this possibility. The reasoning of Reynold v. Smith does, however, strongly point to the outcome that SCOTUS would find such an exclusion to be unconstitutional, because as that ruling says at the top of §II,
Undeniably, the Constitution of the United States protects the right
of all qualified citizens to vote, in state as well as in federal,
Being a minor or a felon are the only two recognized grounds for depriving a citizen of the right to vote in Congressional elections in their state.