As the other comment says, LM Studio is probably the easiest tool. Once you’ve got it installed it’s trivial to add new models. Try some out and see what works best for you. Your hardware will be a limit on what you can run though, so keep that in mind.
Could you point me to a model to do that and instructions on how get it up and running?
Cethin@lemmy.zip 4 days ago
null_dot@lemmy.dbzer0.com 4 days ago
I dont have the hardware so I’m using “open web ui” to run queries on models accessible via huggingface API.
Works really well. I haven’t invested the time to understand how to use workspaces, which allow you to tune models, but aparently its doable.
FauxLiving@lemmy.world 5 days ago
I’m using Deepseek R1 (8B) and Gemma 3 (12B), installed using LM Studio (which pulls directly from Hugging Face).