Have you heard of Homeassistant? It’s a self-hosted smart home solution that fills a lot of the gaps left by the most smart home tech. They’ve recently added and refined support for various different voice assistants, some of which run completely on your hardware. I have found they have great community support for this project and you can also buy their hardware if you don’t feel like tinkering on a Raspberry Pi or VM. The best thing (IMHO) about Homeassistant is that it is FOSS.

Homeassistant Voice Control