Is it possible to forbid linking to a site?
No. It would be quite childish and out of touch to aspire to that level of control. Publishing a website/page for permanent, wide open access is inconsistent with prohibiting accessing it from certain venues, such as hypertext links.
The prohibition of linking to a site is an attempt to enforce a certain degree of selective privacy. As such, it would not be enforced by U.S. courts, and it would be laughable if other jurisdictions proceeded differently. In United States v. Forrester, 512 F.3d 500, 510 (9th Cir. 2008), the court wrote that
e-mail and Internet users have no expectation of privacy in the
to/from addresses of their messages or the IP addresses of the
websites they visit because they should know that this information is
provided to and used by Internet service providers for the specific
purpose of directing the routing of information.
That rationale is equally --or perhaps even more-- applicable to a publisher, since in this case browsers and/or the publisher himself post(s) his URL (the equivalent of a "from" address of emails) "for the specific purpose of directing the routing of information".
A hyperlink is an address, so it would be similar, I believe, to
disallowing someone to list an address (on a map for instance).
That analogy is inadequate because it overlooks essential differences between a web address and a physical (be it home, office, etc.) address. A web address is used for obtaining information which the initial publisher deliberately makes available to the public. By contrast, the act of having one's physical address registered somewhere else serves no such purpose whatsoever.
Another important difference between a hyperlink and a physical address is that "consuming" a hyperlink simply cannot annoy or harm the initial publisher (except in the extreme scenario of Denial-of-Service attacks), whereas a common knowledge of a person's address may make that person vulnerable to harassment, trespassing, larceny, and other unlawful acts.
A better analogy with hypertext links would be bibliographical references, since both are types of text strings for directing the consumer to a (or "the") source of information. Neither text string causes detriment to the author/source of that information. Can you imagine if bibliographical references were forbidden by statute or by the author of that information?
The mere difference that a bibliographical reference needs to be copied/pasted, whereas it suffices to click on a hypertext link, cannot permit treating the permissibility of hypertext links any differently than the permissibility of bibliographical references.