Considering that predicting the next word from context is the one thing LLMs are really good at, I just don’t understand how none of these developments have found their way into predictive keyboards.

The problem is that LLMs require a considerable amount of computing power to run, unlike the simple markov chain predictions that keyboards use. You could use a cloud-based service like ChatGPT or something, but most people wouldn’t want their keyboards to send all their keystrokes to a remote server… and even if they didn’t know or care, the response time wouldn’t be good enough for real-time predictions.

Now smartphone SoC makers like Qualcomm have started adding NPUs (neural processing units) with their latest chips (such as the SD8 Gen 3, featured in phones this year), but it’s going to take a while before devices with NPUs become commonplace, and it’ll take a while for developers to start making/updating apps that can make use of it.

But yeah the good news is that it is coming, it’s only a matter of “when” - I suspect it won’t be long before the likes of SwiftKey start to take advantage of this.