Comment on Recommendations on running GPTs on Asahi - M1 Ultra?
Guenther_Amanita@feddit.de 5 months agoAlright, interesting… As I said, I’m no expert or anything and this was just my noob optinion.
Thank you for the correction and further resources!
plsnotracking@lemmy.world 5 months ago
Thank you for the discussion folks, I’ll try out Llama.cpp and report back.
I also saw the Neural Engine Stuff hasn’t been merged into main kernel yet but it’s available as a separate out of tree patch. Hopefully merging that will help with more model support? (Pure guesswork)
I also saw that PCL stuff isn’t ready yet and u/marcan42 said it’s WIP that also might be helpful in getting better model support because read somewhere that metal isn’t going to be a part of the Asahi kernel ever(?)
I’m no expert at any of this but hopefully we’ll be able to run some sort of GPTs locally someday.
Good luck.