Comment on Anubis is awesome! Stopping (AI)crawlbots
blob42@lemmy.ml 20 hours agoRight I must have just blanket banned the whole /8 to be sure alibaba cloud is included. Did some time ago so I forgot
Comment on Anubis is awesome! Stopping (AI)crawlbots
blob42@lemmy.ml 20 hours agoRight I must have just blanket banned the whole /8 to be sure alibaba cloud is included. Did some time ago so I forgot
Cozog@feddit.dk 16 hours ago
When I blocked Alibaba, the AI crawlers immediately started coming from a different cloud provider (Huawei, I believe), and when I blocked that, it happened again. Eventually the crawlers started coming from North American and then European cloud providers.
Due to lack of time to change my setup to accommodate Anubis, I had to temporarily move my site behind Cloudflare (where it sadly still is).