May there exist a non-voting board of directors member in the United States? Or by definition are the board of directors all voting?
Yes. A company can run itself however it sees fit - if it wants to have non-voting directors, it can.
The relevant provision is s134 of the Corporations Act 2001:
Internal management of companies
A company's internal management may be governed by provisions of this Act that apply to the company as replaceable rules, by a constitution or by a combination of both.
Note: There are additional rules about internal management in ordinary provisions of this Act and also in the common law.
The relevant replaceable rules for directors mention “directors entitled to vote” so they clearly contemplate directors who are not entitled to vote. By default disqualification arises from conflicts of interest but the company is free to decide other causes of disqualification.