All the admins with database access can, yes.

This is the fediverse, everything has to be public and auditable otherwise vote manipulation would be easy.

If you want to really be anonymous, your best bet is to use multiple accounts on multiple instances. If you upvote sketchy stuff, use the anonymous account, and otherwise, use your regular more public account.