Can you be trespassing on public property for no reason at all?
Yes.
When the government owns property, it can direct you to leave for any reason (even a legally invalid reason), and you are trespassing if you don't leave.
When you are present on land you don't own with permission, but without a lease, you have a "license" to be there which is a contract-like right and is not a property right.
A license doesn't give you the right to stay on the property over the objections of the owner or an agent of the owner.
You might be able to receive money damages for an improper termination of your license to be present at the property (e.g. if you are told for no reason to leave a movie after paying for a ticket), but you don't have the right to simply stay there. If you stay there over the objections of the owner or the owner's agent, you are trespassing.
The law applicable to government property owners and private property owners is basically the same in this regard.
Realistically, on government property, furthermore, the standard by which the government employee may legally terminate your license to be there and exclude you from the property is low.
Basically, it must merely not violate any constitutional right you may have, and you do not have a constitutional right to be present on government owned property, except in a quite narrow subset of cases (e.g. the "town square").
More exactly, you do not have a right to be on government property per se, but you can't be excluded from it for a constitutionally impermissible reason. The government gets to decide what parts of property it owns are available to the general public and for what content-neutral purposes.
Thus, the right to be present on government owned "public" property (which doesn't include private areas of government owned property) can be subjected to reasonable and content-neutral time, place, and manner restrictions.
For example, a town could legally decide that the town square is closed from midnight to eight a.m. every day.
The quoted material from the case Chicago v. Morales, 527 U.S. 41, 53-54 (1999) cited in the answer by bdb484 is narrower than a plain reading out of context would suggest. In that quotation, the term "public place" is being used in a sense much more restrictive than in the broader sense of property that is merely government owned. It is referring to places where the government has expressly or implicitly allowed members of the general public to be present on land that it owns (as opposed, for example, to a government office area of a building, or a maintenance facility in a government owned park, or a conservation area in a government owned park). This narrow sense of the word resolves what would otherwise seem to be a contradiction in the law.
But, the government has the authority to make something that once was a public place into a non-public place going forward.
For example, historically, the Civic Center park in front of the capitol in Denver, Colorado has been a public place. But, the government can and did close it off to the public for many months for maintenance and out of public health concerns when heavy use of it by homeless people and drug dealers caused the premises to be seriously damaged and created a public health risk from it being used to dispose of dirty, used, injection-drug syringes, and for people to defecate.
A Hypothetical
Suppose that Chris is the sole librarian in one of the towns of College Corner, which is on the Ohio-Indiana border, which are in different time zones. In that capacity, Chris has the authority to set library policies including the hours of the library and the rules for its use without the approval of anyone else.
Chris has a hot date at 5:30 p.m. But, at lunch time, Chris learned that the hot date was at 5:30 p.m. in Ohio and not an hour later at 5:30 p.m. in Indiana, but the library's official closing time is 5 p.m. in Indiana, because Ohio was observing daylight savings time, but Indian was not, at the time when this happened.
Chris, as the sole government official in charge of the library, decides to close the library before its posted closing time at 4:00 p.m. Indiana time (5 p.m. Ohio time) in order to be able to make it to the hot date.
Chris quietly asks everyone left in the library to leave at 4 p.m. Indiana time, and everyone but you does. But you, who are homeless, really want to stay the extra hour before getting out in the cold and finding a bridge to sleep under, so you refuse to leave saying that the library is still open until 5 p.m. Indiana time, as stated in its posted hours.
Chris orders you to leave and warns you that he is calling the police to remove you if you don't do so voluntarily. The police arrive and restate the complaint of Chris. The police arrest you for trespassing and you are charged with this crime in the appropriate court by the appropriate prosecuting attorney.
You had done nothing wrong whatsoever prior to being asked to leave and refusing to do so. But, Chris has not violated your constitutional rights by ordering you removed for some unconstitutional reason. Chris then goes on the hot date; it is love at first sight, and Chris gets married the next week. As an apology for putting you out for the sole convenience of Chris, Chris invites you to the wedding.
Do you have a valid defense to the criminal trespassing charge on the grounds that the librarian's actions were unconstitutional? No. You might have a "good faith claim of right" defense, however, to the criminal charges.
Do you have a valid claim for money damages for a violation of your constitutional rights? No.
Indeed, ordering you removed for "no reason" that has anything to do with your conduct, as in this case, is probably more likely to be legal and constitutional, than having you removed for "some reason" other than just "because I the librarian say so" that doesn't involve wrongdoing on your part.
A Variation In The Hypothetical
If instead, your were being ordered to leave the library because you were wearing an National Rifle Association cap (and the library didn't prohibit wearing caps), this affirmative reason, which is contrary to the First Amendment freedom of expression, would be a violation of your constitutional rights, which would definitely be a basis for a civil lawsuit against the librarian and police involved in you being arrested.
I don't know the details of constitutional defenses in criminal law well enough to know if the violation of your constitutional rights would be a valid defense to the criminal trespassing charge (or a failure to obey a police officer's order to leave charge) resulting from you failing to leave in that circumstance, and I can see legitimate arguments both ways. This is also a situation where the non-constitutional claim of right defense to a trespassing charge would be a strong one.