Comment on Anubis is awesome! Stopping (AI)crawlbots

<- View Parent
yetAnotherUser@discuss.tchncs.de ⁨3⁩ ⁨weeks⁩ ago

Plus even if they were to implement those features, the challenges would still get increasingly harder the more bot-like a scraper behaves.

You can’t prevent scraping entirely but you can certainly prevent scraping that behaves like a DOS attack.

source
Sort:hotnewtop