Comment on How to use GPUs over multiple computers for local AI?

• reptar@lemmy.world ⁨1⁩ ⁨week⁩ ago

  Hey I built a micro -atx epyc for work that has tons of pcie slots. Pretty sure it was an ASRock (or ASRack). I can find the details tomorrow if you’d like. Just let me know!

  source

  • marauding_gibberish142@lemmy.dbzer0.com ⁨1⁩ ⁨week⁩ ago

    Thank you, and that highlights the problem - I don’t see any affordable options (around $200 or so for a motherboard + CPU combo) for a lot of PCIe lanes other than purchasing Frankenstein boards from Aliexpress. Which isn’t going to be a thing for much longer with tariffs, so I’m looking elsewhere

    source

    • reptar@lemmy.world ⁨1⁩ ⁨week⁩ ago

      Yes, I inadvertently emphasized your challenge :-/

      source
• litchralee@sh.itjust.works ⁨1⁩ ⁨week⁩ ago

  Prior-gen Epyc boards show up on eBay from time to time, often as CPU+mono bundles from Chinese datacenters that are upgrading to latest gen. These can be had for a deal, if they’re still available, and would provide PCIe lanes for days.

  source

  • BombOmOm@lemmy.world ⁨1⁩ ⁨week⁩ ago

    Yeah, forgot to mention Threadripper as well for lots of PCIe lanes. Here is one that has 4 x16 slots. And, note, I am not endorsing that specific listing. I did very minimal research on that listing, just using it as an example.

    source

    • marauding_gibberish142@lemmy.dbzer0.com ⁨1⁩ ⁨week⁩ ago

      Thanks for the tip on x670, I’ll take a look

      source
    • marauding_gibberish142@lemmy.dbzer0.com ⁨1⁩ ⁨week⁩ ago

      I see. I must be doing something wrong because the only ones I found were over $1000 on eBay. Do you have any tips/favoured listings?

      source

      • BombOmOm@lemmy.world ⁨1⁩ ⁨week⁩ ago

        All I did for that one was search “Threadripper” and look at the pictures for ones with 4x x16 slots that were not hella expensive.

        There are technically filters for that, but, I don’t trust people to list their things correctly.

        source
• just_another_person@lemmy.world ⁨1⁩ ⁨week⁩ ago

  Wow, so you want to use inefficient models super cheap. I guarantee nobody has ever thought of this before. Good move coming to Lemmy for tips on how to do so. I bet you’re the next Sam Altman 🤣

  source

  • marauding_gibberish142@lemmy.dbzer0.com ⁨1⁩ ⁨week⁩ ago

    I don’t understand your point, but I was going to use 4 GPUs (something like used 3090s when they get cheaper or the Arc B580s) to run the smaller models like Mistral small.

    source