Because the idea that there are vegans and vegetarians is an implied accusation of guilt to people that eat meat and animal products that are the direct result of animal suffering. Some people that don’t eat animal products might be militant or preachy, but I’ve never met one. In fact I have only ever met the inverse, people that constantly berate people for being vegan or vegetarian. Having been one myself, I can say that for me and people I know it is the same reason religious people hate atheists; their existence implies that people that eat meat are wrong and therefore bad people. Studies have confirmed this. Here is an article from the BBC that goes over some of the studies.