Comment on How to use GPUs over multiple computers for local AI?
BombOmOm@lemmy.world 1 week ago
Most GPUs don’t need a full x16 slot to run at full speed. There are motherboards out there which will give you 3 or 4 slots of x8 electrical (x16 physical). I would look into those.
marauding_gibberish142@lemmy.dbzer0.com 1 week ago
Aren’t Epyc boards really expensive? I was going to buy 3-4 used computers and stuff a GPU in each.
Are there motherboards on the used market that can run the E5-2600 V4 series CPUs and have multiple PCIe Xi slots? The only ones I found were super expensive/esoteric.
reptar@lemmy.world 1 week ago
Hey I built a micro -atx epyc for work that has tons of pcie slots. Pretty sure it was an ASRock (or ASRack). I can find the details tomorrow if you’d like. Just let me know!
marauding_gibberish142@lemmy.dbzer0.com 1 week ago
Thank you, and that highlights the problem - I don’t see any affordable options (around $200 or so for a motherboard + CPU combo) for a lot of PCIe lanes other than purchasing Frankenstein boards from Aliexpress. Which isn’t going to be a thing for much longer with tariffs, so I’m looking elsewhere
reptar@lemmy.world 1 week ago
Yes, I inadvertently emphasized your challenge :-/
litchralee@sh.itjust.works 1 week ago
Prior-gen Epyc boards show up on eBay from time to time, often as CPU+mono bundles from Chinese datacenters that are upgrading to latest gen. These can be had for a deal, if they’re still available, and would provide PCIe lanes for days.
BombOmOm@lemmy.world 1 week ago
Yeah, forgot to mention Threadripper as well for lots of PCIe lanes. Here is one that has 4 x16 slots. And, note, I am not endorsing that specific listing. I did very minimal research on that listing, just using it as an example.
marauding_gibberish142@lemmy.dbzer0.com 1 week ago
Thanks for the tip on x670, I’ll take a look
marauding_gibberish142@lemmy.dbzer0.com 1 week ago
I see. I must be doing something wrong because the only ones I found were over $1000 on eBay. Do you have any tips/favoured listings?
just_another_person@lemmy.world 1 week ago
Wow, so you want to use inefficient models super cheap. I guarantee nobody has ever thought of this before. Good move coming to Lemmy for tips on how to do so. I bet you’re the next Sam Altman 🤣
marauding_gibberish142@lemmy.dbzer0.com 1 week ago
I don’t understand your point, but I was going to use 4 GPUs (something like used 3090s when they get cheaper or the Arc B580s) to run the smaller models like Mistral small.