Open Menu
AllLocalCommunitiesAbout
lotide
AllLocalCommunitiesAbout
Login

Some local LLMs tested on an average gaming PC

⁨67⁩ ⁨likes⁩

Submitted ⁨⁨1⁩ ⁨month⁩ ago⁩ by ⁨donatas@feddit.uk⁩ to ⁨selfhosted@lemmy.world⁩

https://github.com/donatas-xyz/AI/discussions/1

source

Comments

Sort:hotnewtop
  • slazer2au@lemmy.world ⁨1⁩ ⁨month⁩ ago

    Average gaming PC.

    128GB ram and a 4070 12GB. Doesn’t sound average.

    source
    • Duke_Nukem_1990@feddit.org ⁨1⁩ ⁨month⁩ ago

      Yeah, not even a 5090? Pf, nice Dwarf Fortress machine I guess.

      source
      • superkret@feddit.org ⁨1⁩ ⁨month⁩ ago

        A dwarf Fortress machine would have an i9-14900KS, 192GB DDR5 RAM and no graphics card.

        source
        • -> View More Comments
    • iAmTheTot@sh.itjust.works ⁨1⁩ ⁨month⁩ ago

      No kidding. Steam survey definitely does not agree with this definition of average lmao

      source
    • archonet@lemy.lol ⁨1⁩ ⁨month⁩ ago

      I was gonna say, 4x the amount of RAM I have, made me feel inadequate.

      source
      • surewhynotlem@lemmy.world ⁨1⁩ ⁨month⁩ ago

        is my 2080 now considered potato gaming?"

        I’m going to keep using this 1070 until it fails it I get bored of Skyrim.

        source
        • -> View More Comments
      • donatas@feddit.uk ⁨1⁩ ⁨month⁩ ago

        128GB won’t make any difference in gaming when compared to, say, 32GB, so it’s just to fit larger LLMs into memory. However, 0.5t/s is hardly usable for every day tasks, so 32GB will still be more realistic combination with the RTX4070.

        source
        • -> View More Comments
    • False@lemmy.world ⁨1⁩ ⁨month⁩ ago

      I thought I had a lot of RAM with 64

      source
      • Valmond@lemmy.world ⁨1⁩ ⁨month⁩ ago

        Well, you do.

        source
      • sus@programming.dev ⁨1⁩ ⁨month⁩ ago

        64 gigs of ram costs less than $200 these days

        source
  • Blue_Morpho@lemmy.world ⁨1⁩ ⁨month⁩ ago

    I didn’t see tokens/s anywhere?

    source
    • donatas@feddit.uk ⁨1⁩ ⁨month⁩ ago

      Performance/vision based tests have tokens/s at the end of tables.

      source