I think I can create a CNAME record for *.media to point to the Tailscale address of the reverse proxy

This approach, but if you setup your server to advertise subnets you can use your local IP range instead of tailscale’s. Port numbers for individual services would be handled by your reverse proxy, you can setup a subdomain route for each service.

Instead of having to keep track of bookmarks you can use something like homepage on the root of your domain, as a dashboard to navigate to all other services.