That is correct. There is a trick where you can set the source ip to the ipv6 mapped ipv4 ip it originally came from. I have implemented that in a transparent tcp proxy I worked on some years ago, but I am not sure if nginx supports that.

I should look into that actually. It would be useful to me as well.