Comment on Anubis is awesome! Stopping (AI)crawlbots

<- View Parent
Cozog@feddit.dk ⁨4⁩ ⁨days⁩ ago

When I blocked Alibaba, the AI crawlers immediately started coming from a different cloud provider (Huawei, I believe), and when I blocked that, it happened again. Eventually the crawlers started coming from North American and then European cloud providers.

Due to lack of time to change my setup to accommodate Anubis, I had to temporarily move my site behind Cloudflare (where it sadly still is).

source
Sort:hotnewtop