Comment on Stopping the scraper bot menace

Alice@hilariouschaos.com ⁨2⁩ ⁨months⁩ ago

• splinter@hilariouschaos.com ⁨2⁩ ⁨months⁩ ago

  Probably to mine data for training AI. They’re going to scrape anything with user content, not just us.

  source

  • NutinButNet@hilariouschaos.com ⁨2⁩ ⁨months⁩ ago

    Have any other admins of other instances reported similar?

    My first thought was more sinister considering how many other instances seem to hate this instance due to political disagreements and how some of them overlap with the types who dox people for their real identities.

    source

    • splinter@hilariouschaos.com ⁨2⁩ ⁨months⁩ ago

      We don’t really talk, but I think they all struggle with this in some way. Many instances use Cloudflare for example.

      The bots seem to be trying to pull many random posts (e.g. posts that were federated across from other instances)

      My guess is they’re trying to vacuum up the content. It could also be a low-level DoS attack but it feels too light and inefficient for that.

      source

      • NutinButNet@hilariouschaos.com ⁨2⁩ ⁨months⁩ ago

        I am noticing some issues on the third party app Voyager. I am currently accessing via my phone’s browser and I saw the Tollbot prompt for a second and then everything loaded up. But on the Voyager app, it loads for about 10 seconds and then fails. This is what it shows after that imgur.com/a/6oRTUVg trying to pull down and load again results in additional failures.

        This just started this hour, not sure before that as the last time I was successfully on was when I made the previous comment and I didn’t have any issues at that time. Also on the same network as before, so I don’t believe it is my network that is causing this issue either.

        source

        • -> View More Comments
  • Alice@hilariouschaos.com ⁨2⁩ ⁨months⁩ ago

    Hot dam, ok that makes sense. The tool you made pretty much cleaned it all up to l though that’s pretty sick

    source