The problem with putting it behind a VPN is then all your users have to be on VPN.

Self-service IP whitelisting would be easy and let all clients work without trying to hack in a separate VPN client.

The only thing that would suck would be if you were on a mobile link while moving and swapping towers your IP would change so you constantly get kicked off.

But if you were so inclined you could VPN to your own house and your IP would stay the same.