Obviously the police isn't checking all the time that all the speed limit signs are still where they should be, so in practice you would get a speeding ticket, which the police officer would give you with a good conscience. And you might very well think that you missed the sign, and pay the fine without complaining.
If you are sure there was no sign, you could say to that officer "I didn't see any speed limit sign, where was it? " and hopefully he or she would tell you where that sign was supposed to be. Then you might go back, find the sign on the ground, take a photo, take it to the police officer who would then take action to get the sign back up, and would most likely make that speeding ticket invalid.
There are exceptions: A speed limit sign can actually allow you to go faster than you would be allowed without the sign. For example in a town the normal speed limit without any signs might be 30mph, and the sign said 40mph. If the police officer stops you going 45, you have no excuse because without the sign the limit would have been 30. Or you have one sign 30, followed by a sign 40. Same situation if the "40" is taken down. Or the police should have put up repeating signs every two miles, but put them every mile. If one sign is down, they could still be within the legal limits.
And last, assuming the police didn't put the sign up just for fun, there is probably something making it unsafe to go 60mph if there was a sign 40mph. If that is something you should have seen, and doing 60mph was dangerous for reasons you should have seen, then you might get a ticket for driving at an unreasonable speed. Even if there never was a speed sign. You are never allowed to drive at a dangerous speed.