You can be arguably impartial. Youre not banning them for being Nazis you’re banning them for all the stuff Nazis do like the racism. I would also argue you shouldn’t be politically impartial and just ban Nazis because Nazis should not feel welcomed.