You’re right, and it’s infuriating that the AI scrapers are just so lazy/incompetent that they do things like try to scrape every dynamic page of a git repo instead of just cloning it. Similarly, they could just connect over ActivityPub and it wouldn’t have much more overhead than another private instance.

There’s Anubis which uses JavaScript to force browsers to do some work before they can access, but given how unpopular Cloudflare is around here, I imagine there’d be a lot of complaints if it was deployed on every instance.