Yes, for most of them.
Article 42.7 TEU
If a Member State is the victim of armed aggression on its territory, the other Member States shall have towards it an obligation of aid and assistance by all the means in their power, in accordance with Article 51 of the United Nations Charter. This shall not prejudice the specific character of the security and defence policy of certain Member States.
Commitments and cooperation in this area shall be consistent with commitments under the North Atlantic Treaty Organisation, which, for those States which are members of it, remains the foundation of their collective defence and the forum for its implementation.
The "specific character" for "certain Member States" refers to states with a traditional military neutrality, notably Austria, Ireland, and Sweden. Those EU states which are also NATO members would not have that excuse since they agreed, in principle, to use military force to protect others, so they cannot say they are constitutionally incapable of doing so in the EU case.
This is why the EU does not, generally, admit members with open territorial conflicts.
Probably no, for the rest of NATO
The Parties agree that an armed attack against one or more of them in Europe or North America shall be considered an attack against them all and consequently they agree that, if such an armed attack occurs, each of them, in exercise of the right of individual or collective self-defence recognised by Article 51 of the Charter of the United Nations, will assist the Party or Parties so attacked by taking forthwith, individually and in concert with the other Parties, such action as it deems necessary, including the use of armed force, to restore and maintain the security of the North Atlantic area.
This is in a way weaker than the EU clause, "such action as it deems necessary" might allow individual countries to do less than going to war if that is unnecessary. Also, NATO-but-not-EU countries might claim that a Russian attack on EU forces in an EU-but-not-NATO country does not constitute an attack in the treaty area:
For the purpose of Article 5, an armed attack on one or more of the Parties is deemed to include an armed attack:
- on the territory of any of the Parties in Europe or North America, [...]
How to interpret a Russian counter-counter-attack on NATO countries after those counter-attack Russia in accordance with collective defense after a Russian attack on a non-NATO country would become a political issue.
As pointed out in the comments, the practical application of either TEU 42.7 or NATO Article 5 would be intensely political, not just in the "indirect" case but also where it seems to be clear-cut. Nations might rally to the flag or drag their feet, depending on the details. Or even depending on totally unrelated issues where they want to get concessions. Or they might alert their forces and deploy them on different flanks than the one under attack, because that flank feels exposed.