OP said they have an SSD as their boot drive, too, so that would explain why it’s not constantly going. I’m not sure why GPU usage would spike every time the noise happens, though.
Entirely possible it’s a fan bearing instead, but ime if a case fan is going bad you hear it 24/7. And since it’s not the GPU’s fan…