Comment on Is anyone self-hosting their own AI?
boudica@lemmy.world 3 days ago
Llama.cpp running qwen 3.8 on a mini PC with 32GB ram and 8GB vram. Not the must speedy but it works. I only access it within my network and I’m also set up with tailscale.
andrewoidxyz@lemmy.world 3 days ago
So you’re using CPU offloading? Or the model’s Bonsai2 variation?