Comment on [beta] degoog - search engine aggregator

stepan@lemmy.cafe ⁨1⁩ ⁨day⁩ ago

• irmadlad@lemmy.world ⁨1⁩ ⁨day⁩ ago

  Echoing what @thagoat@lemmy.dbzer0.com said, I selfhost it and I only have one user…me.

  source
• thagoat@lemmy.dbzer0.com ⁨1⁩ ⁨day⁩ ago

  I host mine at home and only available on my tailnet. Haven’t had any rate limiting since.

  source

  • fccview@lemmy.world ⁨1⁩ ⁨day⁩ ago

    Yes, I think that’s also key, the more people using the same instance the more likely it’ll be to get banned, but again, rotating proxy is key to this I suppose

    source
• fccview@lemmy.world ⁨1⁩ ⁨day⁩ ago

  Hey! Well, aside from rotating multiple user agents, have a cheeky retry and having a “retry” button next to each engine I also heavily cache results so that for 12 hours the same query will give you the same results from cache. You can always invalidate cache from the settings.

  Another thing you can also do is add multiple proxy that rotate on each request, traffic goes through proxy urls and you don’t get hit with rate limiting (again, proxy settings are in the settings tab - you can find free ones around).

  So far I haven’t been rate limited by google yet, not gonna claim it won’t happen as it probably will, it’s the nature of the tool, but finger crossed so far so good.

  source
• Luminous5481@anarchist.nexus ⁨1⁩ ⁨day⁩ ago

  turn on the limiter and link token, and that’ll stop in SearXNG. this is why most instances don’t work, is they don’t turn on the existing anti-bot measures.

  https://docs.searxng.org/src/searx.botdetection.html

  source

  • Fmstrat@lemmy.world ⁨19⁩ ⁨hours⁩ ago

    Why do these docs look like code? So odd… For my dumb brain trying to absorb this from a phone, is there a doc for what to enable in the Docker container that I’m not seeing?

    source

    • Luminous5481@anarchist.nexus ⁨12⁩ ⁨hours⁩ ago

      there’s some guides online on how to do it, you’ll need to edit some configs and also have a Redis docker up.

      source

      • Fmstrat@lemmy.world ⁨8⁩ ⁨hours⁩ ago

        Ahh definately not my problem after figuring this out (couldn’t find a guide but looked at the source). My instance is private so this isn’t impacting my non-Google results.

        source