Comment on Is anyone self-hosting their own AI?
Nighed@feddit.uk 4 days ago
My server’s old GPU (8gb) can’t run anything useful LLW wise I don’t think.
Have run some stuff locally on my main computer, but nothing setup as a proper service.
Comment on Is anyone self-hosting their own AI?
Nighed@feddit.uk 4 days ago
My server’s old GPU (8gb) can’t run anything useful LLW wise I don’t think.
Have run some stuff locally on my main computer, but nothing setup as a proper service.
Not for complex tasks, really. Gemma4b or similar might be just enough as the backend for a voice assistant replacing Alexa (which I honestly hope, I need to get family members off of that fucking spy devices…). Small web searches might just fit. Or image recognition for Nextcloud Memories, stuff like that.
Immich’s image recognition works fine off the CPU. I did get it running an LMM, but it was useless.