Comment on Intent recognition for HomeAssistant without an LLM?
smiletolerantly@awful.systems 4 days agoThanks for your input! The problem with the LLM approach for me is mostly that I have so many entities, HASS exposing them all (or even the subset of those I really, really want) is already big enough to slow everything to a crawl, and to get bad results from all models I’ve tried. I’ll give the model you mentioned another shot though.
However, I really don’t want to use an LLM for this. It seems brittle and like overkill at the same time. As you said, intent classification is a wee bit older than LLMs.
Unfortunately, the sentence template matching approach alone isn’t sufficient, because quite frequently, the STT is imperfect. With HomeAssistant, currently the intent “turn off all lights” is, for example, not understood if STT produces “turn off all light”. And sure, you can extend the template for that. But what about
- turn of all lights
- turn off wall lights
- turnip off all lights
- off all lights
- off all fights
- …
A human would go “huh? oh, sure, I’ll turn off all lights”. An LLM might as well. But a fuzzy matching / closest Levensthein distance approach should be more than sufficient for this, too.
Basically, I generally like the sentence template approach used by HASS, but it just needs that little bit of additional robustness against imperfections.
Jayjader@jlai.lu 4 days ago
From my understanding of word embeddings (as used by LLMs), you could skip the LLM and directly compare the similarity of what the STT outputs to each task or phrase in a list you have prepared. You’d need to test it out a few times to see what threshold works, but even testing against dozens of phrases should be much faster than spinning up an LLM - and it should be fully deterministic.
smiletolerantly@awful.systems 4 days ago
Yep, that’s the idea! This post basically boils down to “does this exist for HASS already, or do I need to implement it?” and the answer, unfortunately, seems to be the latter.