Comment on Cloudflare blocking AI crawlers

<- View Parent
Prontomomo@lemmy.world ⁨5⁩ ⁨days⁩ ago

I don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling

source
Sort:hotnewtop