Let's say I'm convicted of a felony and sent to state prison. Then, I buy the prison.
Can I just let myself out?
Not legally. Private prisons are operated under a government mandate (typically via a contract with the government).
That said, depending on jurisdiction, there are plenty of scenarios where a prisoner is legally permitted to leave a prison. This can include work release programs, work details controlled by the prison, to visit sick or ill relatives, as part of a rehabilitation program (e.g., to attend an education program), etc.
How much of these forms of release can be manipulated by a private prison likely varies by jurisdiction. Of course, some of the relevant decision makers may have a preference for keeping the prison owner happy, even if the owner has no direct influence.
Mind you, it is probably difficult to get into such a position. I suspect a prison operator would not be sent to their own prison (and would not want to be; prisoners might attack them). Similarly, the government would probably block a jailed felon from taking ownership of their own prison.