Comment on Consumer GPUs to run LLMs
umami_wasbi@lemmy.ml 3 days ago
Using 7900XTX with LMS. Speed are everwhere, driver dependent. With QwQ-32B-Q4_K_M, I got about 20 tok/s, with all VRAM filled.
Comment on Consumer GPUs to run LLMs
umami_wasbi@lemmy.ml 3 days ago
Using 7900XTX with LMS. Speed are everwhere, driver dependent. With QwQ-32B-Q4_K_M, I got about 20 tok/s, with all VRAM filled.
marauding_gibberish142@lemmy.dbzer0.com 3 days ago
I didn’t know that. I thought just one ROCM binary to install, run Ollama and that’s it. Thanks for the explanation