You can run GLM Air on pretty much any gaming desktop with 48GB+ of RAM. Check out ubergarm’s ik_llama.cpp quants on Huggingface; that’s state of the art right now.
Comment on Elon Musk’s Grok Goes Haywire, Boasts About Billionaire’s Pee-Drinking Skills and ‘Blowjob Prowess’
DandomRude@lemmy.world 1 week agoThx for clarifying.
I once tried a community version from huggingface (distilled), which worked quite well even on modest hardware. But that was a while ago. Unfortunately, I haven’t had much time to look into this stuff lately, but I wanted to check that again at some point.
brucethemoose@lemmy.world 1 week ago
brucethemoose@lemmy.world 1 week ago
Also, I’m a quant cooker myself. Say the word, and I can upload an IK quant more tailored for whatever your hardware/aim is.
DandomRude@lemmy.world 1 week ago
Thank you! I might get back to you on that sometime.
brucethemoose@lemmy.world 1 week ago
Do it!
Feel free to spam me if I don’t answer at first. I’m not ignoring you; Lemmy fails to send me reply notifications, sometimes.