It turns out that people on Lemmy are no better responsibly using a downvote button than anywhere else. I think you should have to at least select a reason why you’re downvoting to add some friction - maybe options something like “I don’t like this”, “I disagree”, “This is factually incorrect”, “Spam”, “Abusive language”, etc. Then you can filter out the first two!