Comment on Cloudflare blocking AI crawlers
Prontomomo@lemmy.world 5 days agoI don’t think this blocks crawlers. About 1/5 websites uses cloudflare, the significant thing here’s is that AI scraping is now blocked by default on most of those sites, NOT crawling