Keep in mind that a lot of webpages block traffic from datacenters, as they are trying to protect themselves from AI scrappers. I recently had an issue with OpenAI making thousands of requests to one of my servers.