On you server: tcpdump -ni any tcp port <your server port>

You will know if your traffic reaches your server and to what it has to respond to.

It feels like NAT issue to me. Or DNS :D