ill check robonope out seems promising
Comment on How to combat large amounts of Ai scrapers
fubarx@lemmy.world 1 week ago
If nginx, here’s an open-source blocker/honeypot: github.com/raminf/RoboNope-nginx
If you have it set up to be proxied or hosted by Cloudflare, they have their own solution: blog.cloudflare.com/declaring-your-aindependence-…
DrunkAnRoot@sh.itjust.works 1 week ago
ikidd@lemmy.world 1 week ago
I wonder why that RoboNope doesn’t just make a fail2ban entry for anything that accesses a disallowed url and drop them entirely.