Comment on OpenAI strikes Reddit deal to train its AI on your posts
drmoose@lemmy.world 1 month agoScraping at scale is actually cheaper than buying API access. It’s a massive rising market, try googling “web scraping service” and there are hundreds of services that provide API to scrape any public web page.
BatrickPateman@lemmy.world 1 month ago
Scraping ia nice for static conten, no doubt. But I wonder at what point it is easier to request changes to a developing thread via API than to request the whole page with all nested content over and over to find the new answes in there.
drmoose@lemmy.world 1 month ago
Following a developing thread is a very tiny use case I’d imagine and even then you can just scrape the backend API that is used on the public page for the same results as private API.