Submitted 11 months ago by throws_lemy@lemmy.nz to technology@lemmy.world
https://www.macrumors.com/2023/12/21/apple-ai-researchers-run-llms-iphones/
Everyone likes to trash machine learning because the power requirements are high, but what they don’t realize is that we’re in the very first days of this technology. Every technology that got bundled together into your phone was equally as useless when it was first invented.
Literally once a week, I see some news story about AI researchers delivering an order of magnitude speedup in some aspect of AI inference. The technique described here apparently allows for a 20x speedup on GPU’s.
Whispercpp works off the ML cores on the m series chips. It’s faster than my 1080ti that I have in a server doing the same things–by orders of magnitude. And it sips power.
Purpose built chips can be super powerful for their specific purposes.
Why the hell do we want to encourage people running MLMS on our phones?!! I don’t want to be part of some stupid pyramid scheme nonsense.
Dotēra Phone
HerbaLife Galaxy S30
Haha
😭
Here, you dropped this: /s
/s is for weaklings.
HEAR ME ROAR.
Siri could suck an order of magnetize less and work offline, for starters
Make Siri Great For Once
It was, and then Apple bought it out.
Tbh I’m more excited to see someone do use webnn, webgpu and petals together. Building smaller tighter models is good too.
I don’t understand the innovation, I already run LLMs and stable diffusion on a laptop from 2011.
I have no doubt it could be run on my Android phone.
AceSLS@ani.social 11 months ago
This comment sums it up pretty nicely:
Apple being “inovative” my ass, lmao
cheese_greater@lemmy.world 11 months ago
Cache u inside
lemmylommy@lemmy.world 11 months ago
Well, if that commenter had more than just a vague idea of caching and/or swapping, they would know that the right algorithm can make or break performance.
That paper is not “we invented caching”, but “this is how we make some certain models work well despite constraints imposed by RAM and flash storage.”
It’s a worthy job for an engineer or researcher. Not quite as innovative as the invention of the wheel, but still enough to write a paper on (and read it, if you can manage to understand it).
Hegar@kbin.social 11 months ago
The easiest way to tell that something's not really innovative is if the person describing it uses the word innovative.
HeartyBeast@kbin.social 11 months ago
Can you give an example of something that actually was innovative, that no-one called innovative?
4am@lemm.ee 11 months ago
35 upvotes in the technology community…man you guys really are just all knee-jerk reactionaries and it really knowledgeable tech at all. git gud