Comment on [AIT] what is your selfhosted local LLM stack?

heyeddi@lemmy.world ⁨3⁩ ⁨days⁩ ago

Ollama on an N150 mini PC, 16GB, no GPU, Open WebUI on top. Being honest, 1-3B models are fine for searching my offline Kiwix stuff and 8B works if you don’t mind waiting. I’m turning this into a small box (heyeddi.com/projects/doomsday-box) and I keep going back and forth on whether people even want AI on it, or just the media and offline archives. Would you?

original
Sort:hotnewtop