The funniest thing about your question is defining 'non-whites.'

Back in the day, any Black ancestor made a person Black; that was so slave owners could sell their children by slave mothers. Thomas Jefferson had a slave wife/concubine/rape victim named Sally Hemmings. She was likely his legal wife's half sister. Sally's granddaughter was 1/8th of less African, but it was considered a scandal when she ran off and passed as White.

The longer any family has been in America, the more likely it is that they are 'non-white.'