Comment

Comment on ChatGPT 5 power consumption could be as much as eight times higher than GPT 4 — research institute estimates medium-sized GPT-5 response can consume up to 40 watt-hours of electricity

<- View Parent

EncryptKeeper@lemmy.world ⁨1⁩ ⁨week⁩ ago

AI models require a LOT of VRAM to run. Failing that they need some serious CPU power but it’ll be dog slow.

A consumer model that is only a small fraction of the capability of the latest ChatGPT model would require at least a $2,000+ graphics card, if not more than one.

source

Sort:hotnew top

Corkyskog@sh.itjust.works ⁨1⁩ ⁨week⁩ ago
How slow?

source
- Evono@lemmy.dbzer0.com ⁨1⁩ ⁨week⁩ ago
  Basicly I can run 9b models on my 16gb gpu mostly fine like getting responses of lets say 10 lines in a few seconds.
  
  Bigger models if they don’t outright crash take for the same task then like 5x or 10x longer so long it isn’t even useful anymore
  
  So very worse.
  
  source
- EncryptKeeper@lemmy.world ⁨1⁩ ⁨week⁩ ago
  Like make a query and then go make yourself a sandwich while it spits out a word every other second slow.
  
  There are very small models that can run on mid range graphics cards and all, but it’s not something you’d look at and say “Yeah this does most of what chatGPT does”
  
  source
- gerryflap@feddit.nl ⁨1⁩ ⁨week⁩ ago
  It’s horrendously slow, unusable imo. With the larger DeepSeek distilled models I tried that didn’t fit into VRAM you could easily wait 5 minutes until it was done writing its essay. Compared to just a few seconds when it does. Bit that’s with a RTX 3070 Ti, not something the average ChatGPT user has lying around probably.
  
  source