For some items like glasses it’s very clear why they are pairs; if you can have a reading glass (which is an antiquated way to refer to a handheld magnifying lens, for example) then you can certainly have a pair of reading glasses because it’s the two pieces of glass which are plural.

For trousers there are no certain answers, but I’d suggest it’s very much with with how we conceptualise their function. For 90% of their height trousers are split and cover the legs, of which we have two, only joining right at the top.

For shirts you might think it’s the same (two arms right?) but it’s a completely different story because the primary function of a shirt isn’t to cover the arms but to cover the torso. So it’s singular. And gloves of course are distinct, so it’s back to pairs.