Comment on Would you use a self-hosted, AI-powered search engine for your favorite sites?
CameronDev@programming.dev 1 week ago
I personally have zero interest in AI search, if you mean LLM. The fact that it can make stuff up, also means it can miss stuff as well. Neither are acceptable for a search engine.
If you mean some kind of deterministic algorithm for indexing and searching, then maybe.
Also, attempting to crawl sites locally sounds like a great way to get banned from those sites for looking like a bot.
T156@lemmy.world 1 week ago
I can’t imagine self hosting an LLM-based search engine would be too viable. The hardware demands, even for a relatively small quantised model, are considerable. Doubly so if you don’t have a GPU to accelerate with.
CameronDev@programming.dev 1 week ago
Yeah, absolutely. And running a GPU 24/7 to occasionally search is just a waste of power. I’m not convinced that google and bings AI search makes financial sense either, Google dropped live search (where the results updated as you typed realtime) because it was too expensive, how does LLM search end up cheaper than live search?!
Jakeroxs@sh.itjust.works 1 week ago
You realize the gpu site idle when not actively being used right?
It’d be cheaper if you host it locally, essentially just your normal electricity bill which is the entire point of what op is saying lol.
CameronDev@programming.dev 1 week ago
Idle is low power, not zero power. And it won’t be idle when its scraping and parsing the sites, so depending on how much scraping its doing, it could be significant non-idle energy usage.
JeremyHuntQW12@lemmy.world 1 week ago
You can run Deepseek on a Raspberry Pi.
Senal@programming.dev 1 week ago
At a level you’d need to use for a search engine ?