What are the legal exit options for a person who enters a country illegally (by say crossing a land border) and wants to exit that country with a valid passport?
It certainly depends on the country, but in some cases, the country's immigration authorities do not check the documents of people leaving the country, so there would be no problem. An example of such a country is the USA.
In some cases one can leave by paying a fine, but these fines can be quite high. In other cases, the person might be detained and perhaps imprisoned as punishment for the violation of immigration law.
After the detainment and possible imprisonment, the person would likely be deported back to the country of citizenship, which could force the person's hand to make an asylum application. After being granted asylum, the country might issue a refugee travel document allowing travel abroad.
Alternatively, after release, the person might be allowed to travel out of the country voluntarily, to whatever destination he or she might choose. However, it is unlikely that the authorities would allow this unless the person could establish that the destination would be likely to accept him or her. In other words, the authorities would normally require the person to have a visa for the destination, or a passport that allows visa-free visits there. If the person does not have adequate documents, the destination country would likely refuse entry, in which case there's a good chance that the person would be returned to the authorities of the country he or she is trying to leave, where he or she is unwelcome.