It’s about the findability of the post and correctness of votes. A serial downvoter will diminish both, creating a disconnect between the real quality and the perceived quality.

No those three votes probably won’t make a big difference, but it’s a real problem if this becomes commonplace