There are many ways to do this and you got the right gist, but my recommendation:

• Set up a WireGuard tunnel connecting your VPS and homeserver
• Set up a layer-4 TCP reverse proxy (Nginx’s stream module/Traefik TCP routers/Caddy-L4/HAProxy are all doable) on the VPS
• Use that reverse proxy to route all TCP traffic back to the homeserver’s HTTPS service(s), via the wg tunnel

Here’s a guide that helped me with such a setup: theorangeone.net/…/wireguard-haproxy-gateway/

Wireguard only need one peer to open a silent UDP port, so use the VPS’ IP and no need to portforward your homeserver. There are other more convenient solutions like Tailscale or Pangolin, but being Wireguard-based they all follow the same principle. Lastly this keeps your certs locally for TLS all the way through