Comment on Is anyone self-hosting their own AI?

Rentlar@lemmy.ca ⁨1⁩ ⁨week⁩ ago

Played around with it on my desktop machine. Someone’s distilled variant of Qwen 3.8-27B-Q4_K_M, on a intel12700k + RTX3090 24GB is what I use.

I haven’t adjusted the reasoning down from very high but I mostly use it for trying tweaks to half-finished coding projects I’ve lost motivation to complete many years ago. I used it to develop a very simple JS-free personal website. I have secure shell which I can access from Termux or PuTTY, but sometimes I have the llama.cpp webui insecurely accessible online for one day at a time just for fun.

original
Sort:hotnewtop