Yes, of course you still owe it. There's no logical reason why ceasing to be a citizen should relieve you of existing obligations.
The State Department mentions this explicitly
Persons who wish to renounce U.S. citizenship should be aware of the fact that renunciation of U.S. citizenship may have no effect on their U.S. tax or military service obligations (contact the Internal Revenue Service or U.S. Selective Service for more information). In addition, the act of renouncing U.S. citizenship does not allow persons to avoid possible prosecution for crimes which they may have committed or may commit in the future which violate United States law, or escape the repayment of financial obligations, including child support payments, previously incurred in the United States or incurred as United States citizens abroad.
I think the "may" is just to cover their butts - I can't find any indication of any provision that would forgive tax debts when you renounce.
Indeed, renouncing your citizenship may cause you to owe more tax, because of the expatriation tax. Basically, all your unrealized capital gains are treated as if they were realized and taxed on the day before your expatriation, and you owe capital gains tax on them.