Being able to choose the OS and kernel is also important. I would not want my hypervisor machine to load GPU kernel modules, especially not on an older LTS kernel (which often don’t support the latest hardware). Passing the GPU to a VM ensures stability of the host machine, with the flexibility to choose whatever kernel I need for specific hardware. This alongside running entirely different OSes (like *BSD, Windows :(, etc) is pretty useful for some services.