Comment on Cloudflare to AI Crawlers: Pay or be blocked

drmoose@lemmy.world ⁨5⁩ ⁨days⁩ ago

This has been tried before many times. The problem is that this exchange can never satisfy all of the parties.

Would site host take 0.01$ for a page? If its Walmart.com then they’d happily lose even 0.10$ or more if competitors can’t analyze their products and other perceived IP damages.

For example, let’s assume they do the business math and come out that if it is anything below 5$ is a no deal - what scraper would pay 5$ for a single product page scrape? Maybe openAI can pay that but is this what we want where public scraping is only accessible to billionaires? What if you’re just a user that wants to track Walmart price to build your own budgeting script? Are you paying 5$ on every request?

source
Sort:hotnewtop