Game design is a big part of this too. Particularly first person or other fine camera control feels very bad when mouse movement is lagging.

I agree with what the other commenters are saying too, if it feels awful at 45 fps your 0.1% low frame rate is probably like 10 fps