Comment on Is anyone self-hosting their own AI?
jwiggler@lemmy.dbzer0.com 12 hours ago Pretty much my setup as well. Though I’m using qwen3.8 27b-Q4_K_M under ollama. Everything running with docker. Add nginx and either a domain+port forwarding or zerotier/tailscale you’ve got an app you can get to from outside your house.
Right now not sure how I’m going to leverage it not probably to streamline my self hosted stuff, help with config mgmt and documentation, web page summarization, etc. we’ll see how it does, just got it setup yesterday
robber@lemmy.ml 3 hours ago
In case you are unaway, frieds don’t let friends use ollama. The llama.cpp has become pretty much equally easy to use, plus since you use Docker there’s no need to compile anything.
Oh damn what a useful link. Gonna check out those orchestrators, maybe I can replace my janky ass proxy.
Oh shit I wasn’t aware! Thanks for the heads up, I’ll swap to llama.cpp