Oh shit I wasn’t aware! Thanks for the heads up, I’ll swap to llama.cpp
Comment on Is anyone self-hosting their own AI?
robber@lemmy.ml 2 hours agoIn case you are unaway, frieds don’t let friends use ollama. The llama.cpp has become pretty much equally easy to use, plus since you use Docker there’s no need to compile anything.
jwiggler@lemmy.dbzer0.com 2 hours ago
Oh damn what a useful link. Gonna check out those orchestrators, maybe I can replace my janky ass proxy.