Short answer: If you agree to X subject to condition Y, then condition Y attaches to X but not necessarily to other things like X that you do. So an agreement might apply to just one copy of the software, or just one visit to a website, or just data uploaded/downloaded during a particular time period, depending on the words and circumstances.
Long answer:
It depends on the precise words and circumstances.
Generally speaking, the terms of service for a website will say 'You use the website subject to XYZ.' So, every visit is a new contract, potentially with different terms to the previous contract. This then (I will say this a lot) depends on the words and circumstances:
Example 1: you visited a website on 1 January with version 1 of the terms of service, which contained a provision requiring you to rub your head and pat your tummy at all times during that visit, then that obligation ends as soon as you leave the website.
Example 2: you visited a website on 1 January with version 1 of the terms of service, which contained a provision requiring you to keep any data downloaded from the website during your visit secret. That obligation continues in force. Suppose on 2 January you visit the website again and now version 2 of the terms of service are up, and version 2 omits the secrecy obligation. The data you downloaded on 1 January is still secret but any data you download on 2 January is not.
Then you have software end user licence agreements. It depends on the words and circumstances, but generally speaking the crux of an end user licence agreement is 'You may use the software on condition that XYZ.'
So, suppose version 1.6.1 comes with an end user licence agreement that says you can use the software to develop nuclear weapons, but then version 1.6.2 comes out with an end user licence agreement that prohibits the use of the software to develop nuclear weapons. People who only downloaded version 1.6.1 would be free to use version 1.6.1 to develop nuclear weapons. People who are using version 1.6.2 could not use that version to develop nuclear weapons.
Generally (but this depends on the precise words and circumstances) people who downloaded both versions 1.6.1 and 1.6.2 would be able to use their copy of version 1.6.1 to develop nuclear weapons. You see this in practice where an open source project used to be GPL but then changed to MIT (or vice-versa, or to a proprietary licence, etc) and someone forks the version with the old licence.
But, read the documents. Because there might be something in there that says 'This supersedes all other agreements', 'You agree to comply with any new version of the X licence that Y publishes', etc. It depends on the words and circumstances.
To take the comment in your question: 'Unless this is very well specified, then all privacy is pointless. All a company has to do is introduce an agreement that states they can change the agreement without notice, then insert some heavy handed bs that allows them to do whatever possible to you and your data, and then remove it on the next version.'
Generally speaking a privacy agreement will say 'We take your data on conditions X, Y and Z.' So the old privacy agreement applies to data collected from you before a certain date. But the company will probably have something in the new agreement which says 'This applies to all data collected before and after the commencement of this agreement'. Generally you would be given an opportunity to opt out of the new agreement by cancelling your service. How this works in practice depends on the words and circumstances.
See the pattern? (1) Agreements of this kind attach obligations to specified subject matter, so that's what you look for; and (2) It all depends on the words and circumstances.
(Then there is the question of whether you had notice of the terms, and then there is the question of vitiation of the contract by unconscionable dealing etc, and then there are all kinds of statutory consumer protections and such forth, but I think that's outside the scope of the question.)
