Comment on ChatGPT 5 power consumption could be as much as eight times higher than GPT 4 — research institute estimates medium-sized GPT-5 response can consume up to 40 watt-hours of electricity

• EncryptKeeper@lemmy.world ⁨1⁩ ⁨week⁩ ago

  AI models require a LOT of VRAM to run. Failing that they need some serious CPU power but it’ll be dog slow.

  A consumer model that is only a small fraction of the capability of the latest ChatGPT model would require at least a $2,000+ graphics card, if not more than one.

  source

  • Corkyskog@sh.itjust.works ⁨1⁩ ⁨week⁩ ago

    How slow?

    source

    • Evono@lemmy.dbzer0.com ⁨1⁩ ⁨week⁩ ago

      Basicly I can run 9b models on my 16gb gpu mostly fine like getting responses of lets say 10 lines in a few seconds.

      Bigger models if they don’t outright crash take for the same task then like 5x or 10x longer so long it isn’t even useful anymore

      So very worse.

      source
    • EncryptKeeper@lemmy.world ⁨1⁩ ⁨week⁩ ago

      Like make a query and then go make yourself a sandwich while it spits out a word every other second slow.

      There are very small models that can run on mid range graphics cards and all, but it’s not something you’d look at and say “Yeah this does most of what chatGPT does”

      source
    • gerryflap@feddit.nl ⁨1⁩ ⁨week⁩ ago

      It’s horrendously slow, unusable imo. With the larger DeepSeek distilled models I tried that didn’t fit into VRAM you could easily wait 5 minutes until it was done writing its essay. Compared to just a few seconds when it does. Bit that’s with a RTX 3070 Ti, not something the average ChatGPT user has lying around probably.

      source