Are you using ZFS? I had issues a few years ago where the arc cache would compete with my VMs for RAM, and after it ran out of RAM it would just completely freeze.