Comment on Is anyone self-hosting their own AI?

PumpkinEscobar@lemmy.world ⁨10⁩ ⁨hours⁩ ago

Qwen 3.8 flash next on a 4090, using strata instead of llama.cpp, gives me about 45-55 tokens/sec. Mainly used for coding tasks. The model size is perfect. If the qwen 4.0 models can be a bit less wordy in reasoning but the same size and quality they’ll be really strong.

original
Sort:hotnewtop