Comment on Chad scraper
oatscoop@midwest.social 11 months agoWhen the data is on multiple sites or sources.
API licenses can be expensive, and some sources might not even have an API.
Comment on Chad scraper
oatscoop@midwest.social 11 months agoWhen the data is on multiple sites or sources.
API licenses can be expensive, and some sources might not even have an API.
madcaesar@lemmy.world 11 months ago
I get the concept but a concrete example. What company could possibly want to pay for scraping a site?
Some dude as a hobby I get it, but what, like Amazon will pay some guy to scrape competition prices or something?
Sotuanduso@lemm.ee 11 months ago
Maybe you’ve got a small company involved in toy buying and reselling, and they want to scrape toy postings from ebay etc. so that they can scroll through a database of different postings and sort it by price or estimated profit or whatever.
oatscoop@midwest.social 11 months ago
I can’t imagine data scraping is something companies will quickly admit to, considering the legal issues involved. It was also the norm for a long time – APIs for accessing user generated data is a relatively new thing.
As for a concrete example: companies using chatGPT. A lot of useful data comes from scraping sites that don’t offer an API.