I have a pretty similar setup currently running but I bought a public domain that I use for my certificates.

I used to have a pi-hole as my DNS server where I entered all subdomains and pointed them at the right address, namely my reverse-proxy.

My reverse-proxy, Nginx Proxy Manager, got the certificates from my domain registrar and forwarded the requests to the correct services based on subdomain.