Comment on What's up, selfhosters? - Sunday thread
Got any links for howtos on this?
Sure! I mostly followed this random youtuber’s video for getting Wyoming protocols offloaded (Whisper/Piper), but he didn’t get Ollama to use his GPU: youtu.be/XvbVePuP7NY.
For getting the Nvidia/Docker passthrough, I used this guide: bittenbypython.com/…/install_ollama_openwebui_ubu….
It’s working fairly great at this point!
pezhore@infosec.pub 1 week ago
Sure! I mostly followed this random youtuber’s video for getting Wyoming protocols offloaded (Whisper/Piper), but he didn’t get Ollama to use his GPU: youtu.be/XvbVePuP7NY.
For getting the Nvidia/Docker passthrough, I used this guide: bittenbypython.com/…/install_ollama_openwebui_ubu….
It’s working fairly great at this point!