Phone proximity is used, so if your phone is in proximity to his, the algorythm can note a relationship between his interests and yours- or even the interests of people who also interact with him.

It’s possible his behaviour is learned from a narcissistic parent, or that enough of his customers are involved in learning about narcissism. You also mightve been at a Cafe near a clinic for long enough your phone tried to ping the office wifi.

Phones spy on us in a dozen different ways, mostly pattern recognition. They track location without GPS (by recording wifi pings), and track interests without the microphone. So they can claim they’re not tracking those specific things while still gathering scary amounts of data.