Edit: I didn't notice a that this question was tagged for Canada; this answer is based on U.S. law.
"Must you stop walking" and "can the police detain you for leaving" are different questions.
Must you stop?
I'd expect a lot of variation from state to state, but there are definitely situations in which you must stop.
In Ohio, for instance, an officer who "reasonably suspects" that that you have committed, are committing, will commit, or have witnessed the commission of violent felony, is permitted to stop you and ask for your name, address and date of birth, and it is a crime to refuse to provide that information. R.C. 2921.29.
But at the moment the officer asks you to stop, you're in a tricky position. If you haven't done anything wrong, you'd be inclined to think that the officer has no basis to stop you and that you're justified in walking away. But if someone just called the police and said someone fitting your description just robbed a store two blocks away, the officer has reasonable suspicion that you committed a violent felony, but you have no way of knowing that. This sort of thing happens pretty much all the time.
In the absence of that reasonable suspicion, though, Ohio courts have repeatedly held that it is not obstruction for you to just walk away (or even run!) from the officer.
Can the police detain you for walking away?
Obviously, if you're in a situation where it is a crime to not answer questions, the police can detain you because they just watched you break the law.
But what about when you're within your rights not to answer? The police can still detain you with a Terry stop when they have a reasonable and articulable suspicion that you are committing a crime, or that you just did, or that you're about to. And they can continue that Terry stop until that suspicion is confirmed or dispelled, or until they can't reasonably expect to get anymore information by detaining you.
Based on the facts you described, it seems unlikely that they could legally detain you based on your termination of the conversation. Still, I imagine that there could be circumstances where they might stop someone, ask questions, and then reasonably suspect that the person was engaged in a crime based on his decision to walk away, especially if the person hasn't explicitly invoked his Fifth Amendment right to silence.