Comment

Comment on Everybody’s talking about Mistral, an upstart French challenger to OpenAI

I run it fine on a base model MacBook Air with 8Gb of RAM and absolutely crazy on a 30 GPU cores M2 Max. Didn’t try on my company’s M1 Pro but I will tomorrow.

I use the LMStudio app and download Mistral from there. The heavier model for my beefy Mac and a 3Gb one for the Air.

I tried a lot of models for development purposes and this one blew my mind.

source

Sort:hotnew top

cheese_greater@lemmy.world ⁨11⁩ ⁨months⁩ ago
Seriously? Might have to try it

source
- joneskind@lemmy.world ⁨11⁩ ⁨months⁩ ago
  You download the model and it’s on your computer for as long as you want. The whole point is to be able to use it locally.
  
  source
  - cheese_greater@lemmy.world ⁨11⁩ ⁨months⁩ ago
    So it is entirely local? Schweet! How large is it?
    
    source
    joneskind@lemmy.world ⁨11⁩ ⁨months⁩ ago
    
    So it is entirely local? Absolutely
    
    How large is it? 12 models of quantization, from 3.08GB to 7.70GB
    
    I use mistral-7b-instruct-v0.1.Q3_K_L.gguf 3.82GB on the MBA
    
    Note that it might crash sometimes during computation. Just push the button “reload” then “continue” and the model finish its sentence as if nothing happened. I don’t know if its related to MLStudio (the app using the model) or the model itself though.
    
    source