Comment on Leak confirms OpenAI is preparing ads on ChatGPT for public roll out
AliasAKA@lemmy.world 3 days agoYou can get by surprisingly well on 20b parameter models using a Mac with decent ram or even 8b parameter models that fit on most high end (eg 16gb) model cards. Depends on your use cases but I almost exclusively use smaller local models.
melfie@lemy.lol 3 days ago
I have a RTX 4060 low-profile in my 2U server and am limited to 8GB of VRAM for anything I self-host at the moment. I may consider a larger chassis with a better GPU in the future.