I tried a couple of times with Jen ai and local llama, but somehow does not work that well for me.
But at the same time i have a 9070xt, so, not exactly optimal
Comment on 1U mini PC for AI?
teslasdisciple@lemmy.ca 1 week agoI’m running ai on an old 1080 ti. You can run ai on almost anything, but the less memory you have the smaller (ie. dumber) your models will have to be.
As for the “how”, I use Ollama and Open WebUI. It’s pretty easy to set up.
ZeDoTelhado@lemmy.world 1 week ago
kata1yst@sh.itjust.works 1 week ago
Similar setup here with a 7900xtx, works great and the 20-30k models are honestly pretty good these days. Magistral, Qwen 3 Coder, GPT-OSS are most of what I use