Yeah this is why keeping your self-hosted stuff off the public Internet is a really good move. It's why I put #Tailscale in #Dropserver, so that I can access my stuff without risking my instance to the "background radiation of the internet" (great term!)
