The kind of local/offline LLMs that would work on your phone would not be very good quality. There’s been amazing progress of quantization of LLMs to get them working on weaker GPUs with lower VRAM and CPUs, so maybe it’ll occur, but I’m not an expert.
I also don’t foresee them linking it up to a cloud-based LLM as that’d be a shit load of queries and extremely expensive.
astraeus@programming.dev 6 months ago
OpenAI is probably already handling a significant amount of queries, I think for daily use the LLN should simply initialize a word map based on user history and then update it semi-occasionally, like once a week or two. Most people don’t drastically change their vocabulary in the course of a few weeks
EatYouWell@lemmy.world 6 months ago
We’re talking about orders of magnitude more queries if we start offloading predective text like that.