Comment on Is possible to run a LLM on a mini-pc like the GMKtec K8 and K9?
Tempo@lemmy.ml 4 months ago
They’re Ryzen processors with “AI” accelerators, so an LLM can definitely run on hardware on one of those. Other options are available, like lower powered ARM chipsets (RK3588-based boards) with accelerators that might have half the performance but are far cheaper to run, should be enough for a basic LLM.
The K8 it’s Ryzen, the K9 Intel, money isn’t a problem and it’s not a spending it’s a investment I need it for business, which of these two models would you recommend for a reasonable good LLM and Stable Diffusion?
I’m just looking for the most cost-effective solution.
exu@feditown.com 4 months ago
I don’t know of any project that already supports that AI processor. You’d still be using the CPU and GPU at the moment.