It still has to hit the ISP router?

Depends on the IP. I have a static block, so the other IPs have to hit the public IP and come back, because the router doesn’t hold those IPs, itnjust forwards the traffic.