I used to know a guy who would regularly say “si” instead of “yes” in English conversations. He also wasn’t Spanish/Hispanic. It was super confusing because I thought he was interjecting to see “See…” instead of just responding affirmatively.