Comment on Rabbit R1 AI box revealed to just be an Android app
deafboy@lemmy.world 1 month agoThe best way to do on-device AI would still be a standard SoC. We tend to forget that these mass produced mobile SoCs are modern miracles for the price, despite the crapy software and firmware support from the vendors.
No small startup is going to revolutionize this space unless some kind of new physics is discovered.
Buddahriffic@lemmy.world 1 month ago
I think the plausibility comes from the fact that a specialized AI chip could theoretically outperform a general purpose chip by several orders of magnitude, at least for inference. And I don’t even think it would be difficult to convert a NN design into a chip or that it would need to be made on a bleeding edge node to get that much more performance. The trade off would be that it can only do a single NN (or any NNs that single one could be adjusted to behave identically to, eg to remove a node you could just adjust the weights so that it never triggers).
So I’d say it’s more accurate to put it as “the easiest/cheapest way to do an AI device is to use a standard SoC”, but the best way would be to design a custom chip for it.
AdrianTheFrog@lemmy.world 1 month ago
They’re not a chip manufacturer though, and modern phone processors are already fast enough to do near real time text generation and fast image generation (20 tokens/second llama 2, ~1 second for a distilled SD 1.5, on Snapdragon 8 Gen 3)
Unfortunately, the cheapest phones with that processor seem about $650, and the Rabbit R1 costs $200 and uses a MediaTek Helio P35 from late 2018.
Buddahriffic@lemmy.world 1 month ago
Neither AMD nor nVidia are chip manufacturers. They just design them and send them off to TSMC or Samsung to get made.