Comment

Comment on FOSS infrastructure is under attack by AI companies

I too read Drew DeVault’s article the other day and I’m still wondering how the hell these companies have access to “tens of thousands” of unique IP addresses. Seriously, how the hell do they have access to so many IP addresses that SysAdmins are resorting to banning entire countries to make it stop?

source

Sort:hotnew top

festus@lemmy.ca ⁨2⁩ ⁨weeks⁩ ago
There are residential IP providers that provide services to scrapers, etc. that involves them having thousands of IPs available from the same IP ranges as real users. They route traffic through these IPs via malware, hacked routers, “free” VPN clients, etc. If you block the IP range for one of these addresses you’ll also block real users.

source
- Buelldozer@lemmy.today ⁨2⁩ ⁨weeks⁩ ago
  
  There are residential IP providers that provide services to scrapers, etc. that involves them having thousands of IPs available from the same IP ranges as real users.
  
  Now that makes sense. I hadn’t considered rogue ISPs.
  
  source
  - festus@lemmy.ca ⁨2⁩ ⁨weeks⁩ ago
    It’s not even necessarily the ISPs that are doing it. In many cases they don’t like this because their users start getting blocked on websites; it’s bad actors piggy-packing on legitimate users connections without those users’ knowledge.
    
    source
werefreeatlast@lemmy.world ⁨2⁩ ⁨weeks⁩ ago
If you get something like 156.67.234.6, then 7, then 56 etc just block 156.67.0.0/24

source
- Buelldozer@lemmy.today ⁨2⁩ ⁨weeks⁩ ago
  Sure, network blocking like this has been a thing for decades but it still requires ongoing manual intervention which is what these SysAdmins are complaining about.
  
  source
GreenKnight23@lemmy.world ⁨2⁩ ⁨weeks⁩ ago
[deleted]
source
- Buelldozer@lemmy.today ⁨2⁩ ⁨weeks⁩ ago
  
  fail2ban
  
  I’m familiar with f2b. I even have several clients licensed with the commercial version but it doesn’t fit this use case as there’s no logon failure for it to work with.
  
  I automatically ban any IP that comes from outside the US because there’s literally no reason for anyone outside the US to make requests to my infra.
  
  I have systems setup with geo-blocking but it’s of limited use due to the prevalence of VPNs.
  
  also, use a WAF on a NAT to expose your apps.
  
  This isn’t a solution either because a WAF has no way to know what traffic is bad so it doesn’t know what to block.
  
  source