For anyone who carries a camera (and in the US that’s probably most people), a short interview with the general counsel for the National Press Photographers Association.
While the discussion is largely about people who make a living from their pictures and his examples revolve around confrontations with police, much of this advice applies to all photographers, regardless of the size of the camera or the reason they want to take pictures.
It boils down to, “If you’re out in public, you can take pictures.”
Read the whole thing and stand up for your rights.