Comment on What's up, selfhosters? - Sunday thread
pezhore@infosec.pub 2 weeks agoSure! I mostly followed this random youtuber’s video for getting Wyoming protocols offloaded (Whisper/Piper), but he didn’t get Ollama to use his GPU: youtu.be/XvbVePuP7NY.
For getting the Nvidia/Docker passthrough, I used this guide: bittenbypython.com/…/install_ollama_openwebui_ubu….
It’s working fairly great at this point!