How about pseudonymous as a compromise? Votes could be publicly federated but tied to some uuid instead of the username. That way you still have the same anti spam ability (can see that a user upvoted these things from this instance at this time) but can’t tie it directly to comments or actual user accounts without some extra osint.

It might be theoretically possible to correlate the uuids with an account’s activity and dox the user in some cases, especially with some instances having a single user, but it would be very difficult or impossible to do on larger instances and would add an extra layer. Single user instances would be kind of impossible to make totally private anyway because they can be identified by instance.