all of your beliefs, passions, interests, social expectations, public perception, and many, many other things are impacted by gender

That’s all through societal expectations though, isn’t it? You can take any of these things and find that they’re associated with different genders in different places around the world and different time periods. I don’t think it would make sense to say that a trans person is only trans in a specific place and time.