Comment on Would you use a self-hosted, AI-powered search engine for your favorite sites?

Zwuzelmaus@feddit.org ⁨2⁩ ⁨weeks⁩ ago

• daniskarma@lemmy.dbzer0.com ⁨2⁩ ⁨weeks⁩ ago

  I’ve been thinking for years. Maybe there’s a way to do a collaborative crawler and indexer.

  Get a bunch of people together to create the perfect search engine in these dire times.

  source

  • A_norny_mousse@feddit.org ⁨2⁩ ⁨weeks⁩ ago

    Doesn’t something like this exist already? en.wikipedia.org/wiki/Distributed_search_engine

    source
• wise_pancake@lemmy.ca ⁨2⁩ ⁨weeks⁩ ago

  Potentially that would be a good application of federation and distributed computing

  An Internet archive like distributed tool, that then feeds into local tokenization and indexing.

  Alternatively a centralized service that generates indices and then locally they are queried would save a lot of energy.

  source