Comment on Local AI is one step closer through Mistral-NeMo 12B

• tau@lemmings.world ⁨3⁩ ⁨months⁩ ago

  Just beware that like AMD, Intel GPUs suffer a performance hit when using LLMs because of the CUDA specific optimizations in frameworks like llama.cpp

  source
• j4k3@lemmy.world ⁨3⁩ ⁨months⁩ ago

  Isn’t there an AMD 16 GB card too, like a 7600 or something?

  source

  • PriorityMotif@lemmy.world ⁨3⁩ ⁨months⁩ ago

    It’s in the list I linked on pcpartpicker they’re about $50 more.

    source
  • vividspecter@lemm.ee ⁨3⁩ ⁨months⁩ ago

    There’s many 16gb AMD cards since at least the 6000 series. The 7600 XT is probably want you’re thinking of since the 7600 is only 8gb.

    source