What do you mean by “extremists”? Not all issues, maybe even close to none, can actually have a reasonable moderate position. An example: If there are people who are in favour of abortion and people who want to outlaw abortion, a moderate position must necessarily not support full abortion rights and therefore is immoral. I think that we actually need more (good) extremism, because in many cases, a compromise is simply unacceptable.