You don’t have to explain if you’re attracted to your partner or not. Some people are assholes who can’t be happy unless others are unhappy. Personally I would tell them to fuck off and just not talk to them anymore.