May there exist a non-voting board of directors member in the United States? Or by definition are the board of directors all voting?
Yes. A company can run itself however it sees fit - if it wants to have non-voting directors, it can.
The relevant provision is s134 of the Corporations Act 2001:
Internal management of companies
A company's internal management may be governed by provisions of this Act that apply to the company as replaceable rules, by a constitution or by a combination of both.
Note: There are additional rules about internal management in ordinary provisions of this Act and also in the common law.
The relevant replaceable rules for directors mention “directors entitled to vote” so they clearly contemplate directors who are not entitled to vote. By default disqualification arises from conflicts of interest but the company is free to decide other causes of disqualification.
It would be more common in a non-profit company than a for profit one. Often, a non-voting board member would be a founder of the non-profit who wants a means to interact with and keep apprised of the company's functioning, but due to concerns about conflicts of interest or personal benefit, can't vote on the matters coming before the board.
A non-voting board member also might be an advisor to the company who is on an "advisory board" regarding some matter but not its day to day operations (e.g. an environmental impact or diversity advisory board), or might be, for example, a minor who lacks the legal capacity to serve as a board member but whose insight is appreciated and heeded by the voting board members.