that might actually be the issue. YouTube doesn’t differentiate between upvotes and downvotes really. to the algorithm you’re engaging in the content either way so it’s serving you more to keep you engaged.