If the party to whom the offer letter is issued fails to take any step towards fulfilling its end, can it be argued that there was no meaningful acceptance as for all intents and purposes, no part of the offer that was 'accepted' has been performed?
There was acceptance — the successful bidder communicated its acceptance to the government body and therefore made a promise to pay the deposit. All the elements of a legally binding contract are met:
1. Offer
The offer letter from the government body.
2. Acceptance
Self-evident in your question. The successful bidder communicated that it had accepted the offer from the government body.
3. Consideration
Ordinarily, this would be the deposit. In this case, since the deposit was never sent, the consideration is both the promise (from the bidder) to submit the deposit and the counter promise (from the government party) to give whatever the contract stipulates the bidder will get from the government party.
4. Intent to create legal relations
Not evidenced in the question, but we can generally assume this based on the evidence that both sides are commercial parties.
Therefore, can the government body 'rescind' the offer and argue that there was no contract to begin with because there was no 'acceptance'?
No. The offer was accepted by the bidder so there was 'acceptance', and all the other elements of the contract stand too (there was consideration, etc.).
Conclusion
The bidder likely breached the terms of the contract. This could potentially be a repudiatory breach allowing the government body to choose to end the contract if they wanted (by "withdrawing"). At the very least, the government body would be entitled to sue the bidder for breaching the terms, but there is no legal position of "the contract doesn't exist" from my perspective.