Comment on Consumer GPUs to run LLMs

swelter_spark@reddthat.com ⁨2⁩ ⁨weeks⁩ ago

With an AMD RX 6800 + 32gb DDR4, I can run up to a 34b model at an acceptable speed.

source
Sort:hotnewtop