Short answer, it should work as you describe, everything local, no need to expose anything.

More information is needed to help.

Its either the caddy setup, maybe proxmox internal networking? Or most likely DNS, its almost always DNS.