Comment on Anubis is awesome! Stopping (AI)crawlbots
ikidd@lemmy.world 3 days ago
Something that’s less annoying than Anubis is fail2ban tarpitting the scrapers by putting in a hidden honeypot page link that they follow, and adding the followers to fail2ban.
N0x0n@lemmy.ml 3 days ago
Wow, what a combo ! I guess this would reduce the tarpit’s overall power consumption?
I haven’t looked at your link yet and maybe it already contains my answer, but I wish to customize for how long they are traped into the tarpit before fail2ban kicks in so I can still poison teir AI while saving alot of ressources !!