I want to use this icon on a public website. The Creative Commons license says I must give attribution. I've read here that to give proper attribution, you must put the work's name, author's name, source, and license next to the image. Since I'm using the image as a small icon, it won't work visually to put the attribution next to the image. What should I do to provide attribution without needing to put a bunch of text next to the icon?
Scroll down to see a footer with this kind of link.
This allows the image creator to be credited, which is good, however in this particular case, the probability of the creator exercising their right and suing someone for not following cc-by is pretty small (but of course that's not a reason not to give credit where it is due)
You are not "citing" the image - a citation is a reference to a (scholarly) work from another work, you are attributing a work that you have directly copied.
The licence tells you what you must do:
Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
I highlighted the important point: they tell you what you must do, the how is up to you so long as it is reasonable.