Comment on ELI5 How does chatgpt do its shit?
Dran_Arcana@lemmy.world 8 months agoThe magic sauce is context length within reasonable compute restraints. Phone predictive text has a context length of like 2-3 words, ChatGPT (and other LLMs) have figured out how to do predictions on thousands or tens of thousands of words of context at a time.
doublejay1999@lemmy.world 8 months ago
It’s that why is compute heavy ?
Dran_Arcana@lemmy.world 8 months ago
Correct, and the massive databases of long-length context associations are why you need tens to hundreds of gigabytes worth of ram/vram. Disk would be too slow