Cloudflare usually blocks ‘unknown’ bots, which are basically bots that aren’t search crawlers. Also I’ve got Cloudflare setup to challenge requests for .zip, .tar.gz, or .bundle files, so that it doesn’t affect anyone unless they download over HTTP.

There’s also probably a way to configure something similar in Anubis, if you don’t like a middleman snooping your requests.