currently the votes are public but hidden from Lemmy regular users

Tough. If you think any action you take on a social media platform is private then you shouldn’t be here.

Whether it is private or not isn’t the problem. It’s people assuming any part of is. Behave or suffer. Just like the real world.