Comment on Anubis is awesome! Stopping (AI)crawlbots
azertyfun@sh.itjust.works 4 days agoIf I am not mistaken the 47.0.0.0/8 ip block is for Alibaba cloud
That’s an ARIN block according to Wikipedia so North America, under Northen Telecom until 2010. It does look like Alibaba operate many networks under that /8, but I very much doubt it’s the whole /8 which would be worth a lot; a /16 is apparently worth around $3-4M, so a /8 can be extrapolated to be worth upwards of a billion dollars! I doubt they put all their eggs into that particular basket. So you’re probably matching a lot of innocent North American IPs with this.
blob42@lemmy.ml 4 days ago
Right I must have just blanket banned the whole /8 to be sure alibaba cloud is included. Did some time ago so I forgot
Cozog@feddit.dk 4 days ago
When I blocked Alibaba, the AI crawlers immediately started coming from a different cloud provider (Huawei, I believe), and when I blocked that, it happened again. Eventually the crawlers started coming from North American and then European cloud providers.
Due to lack of time to change my setup to accommodate Anubis, I had to temporarily move my site behind Cloudflare (where it sadly still is).
blob42@lemmy.ml 3 days ago
We need a decentralized community owned cloudflare alternative. Anubis looks on good track.