Are these restrictions set out by the ISP or the dorm?

If you don’t do business with the ISP, then you don’t have to agree to and follow their terms.

So as long as the dorms doesn’t have rules against setting up your own WiFi, then you should be well within your rights to purchase an Internet connection from another provider, but since you are likely not allowed to get your own line installed, you are probably restricted to ISPs that provide a service over the cellular network.

Of course using a cellular connection will give you worse latencies for online games, but at least you can have your own WiFi with low latency for your VR.