I had the same problem as OP. My solution was to port forward to my server but then block connections from all IP addresses accept from my work, which I added to an allowlist.

It’s working well so far, but I think the Cloudflare tunnel is the better option.