Comment on Is anyone self-hosting their own AI?
twilightwolf90@lemmy.world 13 hours agoHow well does it handle concurrency? I could see this as a vast improvement on llamacpp, but I was already speccing out vLLM.
Comment on Is anyone self-hosting their own AI?
twilightwolf90@lemmy.world 13 hours agoHow well does it handle concurrency? I could see this as a vast improvement on llamacpp, but I was already speccing out vLLM.