Yes, lots of people do this. Good idea to remove the battery if possible, or you’ll have a spicy balloon eventually.

VPN (or cloudflare tunnel) is not a bad idea, but its not essential either, my server is publically exposed, and it largely isnt a problem. I only expose port 443 and some specific random high ports though. I wouldnt expose 22 to the internet.

Keep SSH, just dont expose to the internet, its always nice to have multiple ways into a box, incase one is hung or something.