Nested VMs stay performant about three levels deep, so do that as well.
Comment on Ollama not using AMD GPU on Arch Linux
30p87@feddit.org 5 weeks agoWhy not throw that into a VM with VFIO passthrough, plug the GPU in via an external dock and if we are already at abstracting shit away for unnecessary complexity and non-compatibility do all that on windows?
exu@feditown.com 5 weeks ago
possiblylinux127@lemmy.zip 5 weeks ago
Because that is way more complicated?
It is really easy to run ollama in a container.
30p87@feddit.org 5 weeks ago
Really easy to start running it
Then everything goes wrong, from configuration over logs to cuda. And the worst fucking debugging ever.
possiblylinux127@lemmy.zip 5 weeks ago
On Linux you can download Alpaca. I think it is CPU only but it is simpler.
30p87@feddit.org 5 weeks ago
Ollama is simple too, I meant that containers make everything a nightmare to maintain.