Comment on Based on this graph, and this graph alone, guess at what time I completely blocked OpenAI crawlers

<- View Parent
tomjuggler@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

Yes, CloudFlare blocks agents completely if they ignore it’s restrictions. The key is scale - CloudFlare has a birds eye view of traffic patterns across millions of sites and can do statistical analysis to determine who is a bot.

I hate the necessity but it works

source
Sort:hotnewtop