Comment on Consumer GPUs to run LLMs

<- View Parent
Natanox@discuss.tchncs.de ⁨1⁩ ⁨week⁩ ago

They would run with 8x speed each. Should not be too much of a bottleneck though, I don’t expect the performance to suffer noticeably more than 5% from this. Annoying, but getting a CPU+Board with 32 lanes or more would throw off the price/performance ratio.

source
Sort:hotnewtop