Kinda depends on how you are measuring it.

It sounds like you are just going by elected presidents, but quite a few were multiple-term presidents, and those presidents had multiple elections with different opponents.

And sometimes, a losing opponent would go on to win a later election.

Also, no one ran against George Washington, twice.

Out of 59 elections (if you include Washington), I think there have been 49 white guys and one white lady who have lost a US Presidential election at least once (and may or may not have gone on to be President in a later election).

…wikipedia.org/…/List_of_United_States_presidenti…

I am exactly as fun at parties as you would imagine.