Comment on Everybody’s talking about Mistral, an upstart French challenger to OpenAI
bioemerl@kbin.social 11 months agoMixtral GPTQ can run on a 3090
Mistral 7b can run on most modern gpus
Comment on Everybody’s talking about Mistral, an upstart French challenger to OpenAI
bioemerl@kbin.social 11 months agoMixtral GPTQ can run on a 3090
Mistral 7b can run on most modern gpus
joneskind@lemmy.world 11 months ago
Oh boy, I missed Mixtral GPTQ and only tried Mistral 7b
Currently downloading
mixtral-8x7b-v0.1.Q4_K_M.ggufThank you!
bioemerl@kbin.social 11 months ago
Be warned, prompt processing is slow
joneskind@lemmy.world 11 months ago
It is indeed. I’m switching to the instruct model to see if I can get better results for code and documentation.