Is it because I’m (probably) autistic?

No, it’s a basic human decency thing. No man I know would (intentionally) touch without invitation (whether verbal or body language) and if someone does they deserve a smacking.

The most common gray area is tight crowds where some assholes try to get away with groping disguised as “accidents” which is disgusting.