Nested VMs stay performant about three levels deep, so do that as well.
Comment on Ollama not using AMD GPU on Arch Linux
30p87@feddit.org 3 months agoWhy not throw that into a VM with VFIO passthrough, plug the GPU in via an external dock and if we are already at abstracting shit away for unnecessary complexity and non-compatibility do all that on windows?
exu@feditown.com 3 months ago
possiblylinux127@lemmy.zip 3 months ago
Because that is way more complicated?
It is really easy to run ollama in a container.
30p87@feddit.org 3 months ago
Really easy to start running it
Then everything goes wrong, from configuration over logs to cuda. And the worst fucking debugging ever.
possiblylinux127@lemmy.zip 3 months ago
On Linux you can download Alpaca. I think it is CPU only but it is simpler.
30p87@feddit.org 3 months ago
Ollama is simple too, I meant that containers make everything a nightmare to maintain.