Comment on Cloudflare blocking AI crawlers

<- View Parent
Prontomomo@lemmy.world ⁨1⁩ ⁨month⁩ ago

I don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling

source
Sort:hotnewtop