Comment on The air begins to leak out of the overinflated AI bubble

RegalPotoo@lemmy.world ⁨2⁩ ⁨months⁩ ago

• bruhduh@lemmy.world ⁨2⁩ ⁨months⁩ ago

  Search Nvidia p40 24gb on eBay, 200$ each and surprisingly good for selfhosted llm, if you plan to build array of gpus then search for p100 16gb, same price but unlike p40, p100 supports nvlink, and these 16gb is hbm2 memory with 4096bit bandwidth so it’s still competitive in llm field

  source

  • Scipitie@lemmy.dbzer0.com ⁨2⁩ ⁨months⁩ ago

    Lowest price on Ebay for me is 290 Euro :/ The p100 are 200 each though.

    Do you happen to know if I could mix a 3700 with a p100?

    And thanks for the tips!

    source

    • bruhduh@lemmy.world ⁨2⁩ ⁨months⁩ ago

      Ryzen 3700? Or rtx 3070? Please elaborate

      source

      • Scipitie@lemmy.dbzer0.com ⁨2⁩ ⁨months⁩ ago

        Oh sorry, nvidia RTX :) Thanks!

        source

        • -> View More Comments
  • RegalPotoo@lemmy.world ⁨2⁩ ⁨months⁩ ago

    Thanks for the tips! I’m looking for something multi-purpose for LLM/stable diffusion messing about + transcoder for jellyfin - I’m guessing that there isn’t really a sweet spot for those 3. I don’t really have room or power budget for 2 cards, so I guess a P40 is probably the best bet?

    source

    • bruhduh@lemmy.world ⁨2⁩ ⁨months⁩ ago

      Try ryzen 8700g integrated gpu for transcoding since it supports av1 and these p series gpus for llm/stable diffusion, would be a good mix i think

      source

      • RegalPotoo@lemmy.world ⁨2⁩ ⁨months⁩ ago

        8700g

        Hah, I’ve pretty recently picked up an Epyc 7452, so not really looking for a new platform right now.

        The Arc cards are interesting, will keep those in mind

        source
  • PersnickityPenguin@lemm.ee ⁨2⁩ ⁨months⁩ ago

    Can it run crysis?

    source

    • bruhduh@lemmy.world ⁨2⁩ ⁨months⁩ ago

      How about cyberpunk?

      source

      • iAvicenna@lemmy.world ⁨2⁩ ⁨months⁩ ago

        I ran doom on a GPU!

        source

        • -> View More Comments
  • Gormadt@lemmy.blahaj.zone ⁨2⁩ ⁨months⁩ ago

    Personally I don’t much for the LLM stuff, I’m more curious how they perform in Blender.

    source

    • utopiah@lemmy.world ⁨2⁩ ⁨months⁩ ago

      Interesting, I did try a bit of remote rendering on Blender (just to learn how to use via CLI) so that makes me wonder who is indeed scrapping the bottom of the barrel of “old” hardware and what they are using for. Maybe somebody is renting old GPUs for render farms, maybe other tasks, any pointer of such a trend?

      source
  • RegalPotoo@lemmy.world ⁨2⁩ ⁨months⁩ ago

    Digging into it a bit more, it seems like I might be better off getting a 12gb 3060 - similar price point, but much newer silicon

    source

    • bruhduh@lemmy.world ⁨2⁩ ⁨months⁩ ago

      It depends, if you want to run llm data center gpus are better, if you want to run general purpose tasks then newer silicon is better, in my case i prefer build to offload tasks, since I’m daily driving linux, my dream build is main gpu is amd rx 7600xt 16gb, Nvidia p40 for llms and ryzen 8700g 780m igpu for transcoding and light tasks, that way you’ll have your usual home pc that also serves as a server in the background while being used

      source