Comment on Is anyone self-hosting their own AI?
ProperlyProperTea@lemmy.ml 1 week ago
Last night I managed to run Qwen Flash Next iq2 XS using Strata in docker on a VM
I passed through my 3090 64GB of DDR4 12 vcores of my 5960x
Managed 90t/s
I’ll probably try a larger version soon but QFN already seems a lot more capable than 3.6 35b A3B, and runs better than Qwen 3.8 dense in llama.cpp