This might work well with a separate per-user random secret value instead of the password.

Overall the vote privacy issue is a tough dilemma for me.