Ok so This is a bit outside my Caddy knowledge base, but I remember reading about the Caddy RealIP module: github.com/captncraig/caddy-realip. Sounds like maybe that would be something to investigate.