They make one request per IP. Rate limit per IP does nothing.
Comment on Black Mirror AI
MonkderVierte@lemmy.ml 5 weeks ago
Btw, how about limiting clicks per second/minute, against distributed scraping? A user who clicks more than 3 links per second is not a person. Neither, if they do 50 in a minute. And if they are then blocked and switch to the next, it’s still limited in bandwith they can occupy.
JadedBlueEyes@programming.dev 5 weeks ago
MonkderVierte@lemmy.ml 5 weeks ago
Ah, one request, then the next IP doing one and so on, rotating? I mean, they don’t have unlimited adresses. Is there no way to group them together to an observable group?
edinbruh@feddit.it 5 weeks ago
There’s always Anubis 🤷
Anyway, what if they are backed by some big Chinese corporation with some /32 ipv6 and some /16 ipv4? It’s not that unreasonable
JackbyDev@programming.dev 5 weeks ago
No, I don’t think blocking IP ranges will be effective (except in very specific scenarios). See this comment referencing a blog post about this happening and the traffic was coming from a variety of residential IP allocations. lemm.ee/comment/20684186
letsgo@lemm.ee 5 weeks ago
I click links frequently and I’m not a web crawler. Example: get search results, open several likely looking possibilities (only takes a few seconds), then look through each one for a reasonable understanding of the subject that isn’t limited to one person’s bias and/or mistakes. It’s not just search results; I do this on Lemmy too, and when I’m shopping.
MonkderVierte@lemmy.ml 5 weeks ago
Ok, same, make it 5 or 10. Since i use Tree Style Tabs and Auto Tab Discard, i do get a temporary block in some webshops, if i load (not just open) too much tabs in too short time. Probably a CDN thing.
Opisek@lemmy.world 5 weeks ago
Would you mind explaining your workflow with these tree style tabs? I am having a hard time picturing how they are used in practice and what benefits they bring.
MonkderVierte@lemmy.ml 4 weeks ago
Sure, basically extended grouping. Middle-click on link configured to open tab as lower-level tab. And to collapse trees, if the next is opened.
Here with my own Material-esque theme.
Image