Submitted 1 week ago by tofu@lemmy.nocturnal.garden to selfhosted@lemmy.world
https://vulpinecitrus.info/blog/guarding-git-forge-ai-scrapers/?ref=selfh.st
Cross posted from: lemmy.nocturnal.garden/post/407947
Self-hosting anything that is deemed “content” openly on the web in 2025 is a battle of attrition between you and forces who are able to buy tens of thousands of proxies to ruin your service for data they can resell.
This is depressing. Profoundly depressing.
Sigh
Good read, I use crowdsec to block most of the traffic considered as malicious (which tend to overlap with scrappers), but I should look into Locaine to feed garbage to the remaining ones instead of throttling.
it’s [iocaine)(docs.rs/crate/iocaine/latest) not Locaine, tripped me up at first as well.
I would like to know how well iocaines spanky new redirection module works compared to Anubis.
If nothing else, to see if throwing Anubis in front of iocaine is still a worthwhile idea.
Build your own captcha, there’s just no other way to be sure it’s human traffic with prebuilt solutions :(
Honesty we need a POW system built into http.
POW built in to the web spec would be hell. Making every single device in the world do that extra bit of work would noticeably affect energy use across the planet.
Lemmchen@feddit.org 3 days ago
her.esy.fun/posts/…/index.html
Also: iocaine.madhouse-project.org