I am AFAB - been wearing mens pants for years and I don’t have a problem fitting into them. Probably largely depends on body shape and even then, taking measurements and matching it to the correct size usually works out.

My crackpot conspiracy theory is that women’s pants pockets are smaller in order to force women to buy and use purses.