I loved scraping until my ip was blocked for botting lol. I know there’s ways around it it’s just work though
Comment on Chad scraper
bill_1992@lemmy.world 11 months ago
Everyone loves the idea of scraping, no one likes maintaining scrapers that break once a week because the CSS or HTML changed.
camr_on@lemmy.world 11 months ago
pennomi@lemmy.world 11 months ago
I successfully scraped millions of Amazon product listings simply by routing through TOR and cycling the exit node every 10 seconds.
camr_on@lemmy.world 11 months ago
That’s a good idea right there, I like that
AlecStewart1st@lemmy.world 11 months ago
This guy scrapes
aBundleOfFerrets@sh.itjust.works 11 months ago
lmao, yeah, get all the exit nodes banned from amazon.
pennomi@lemmy.world 11 months ago
That’s the neat thing, it wouldn’t because traffic only spikes for 10s on any particular node. It perfectly blends into the background noise.
Touching_Grass@lemmy.world 11 months ago
You guys use IP’S?
camr_on@lemmy.world 11 months ago
I’m coding baby’s first bot over here lol, I could probably do better
synae@lemmy.sdf.org 11 months ago
Token ring for me baybeee
dangblingus@lemmy.world 11 months ago
Or in the case of wikipedia, every table on successive pages for sequential data is formatted differently.
Matriks404@lemmy.world 11 months ago
Just use AI to make changes ¯_(ツ)_/¯
anarchy79@lemmy.world 11 months ago
Here take these: \\
Matriks404@lemmy.world 11 months ago
¯_(ツ)_/¯\\ Thanks
DigitalPaperTrail@kbin.social 11 months ago
spite can be a great motivator, though
Anonymousllama@lemmy.world 11 months ago
This one. One of the best motivators. Sense of satisfaction when you get it working and you feel unstoppable (until the next subtle changes happens anyway)
archomrade@midwest.social 11 months ago
I feel this