The reverse proxy only listens on port 80 and 443, so yes, all your services will be accessible through just one/two ports.

The reverse proxy will parse the http request headers and ask the appropriate upstream service (e.g. jellyfin) on localhost:12345 what it should send as a reply. Yes, this means that you need to have a http header so that the reverse proxy can differentiate the services. You don’t need to buy a domain for that, you can use iPhone to make your made up domain map to a local IP address, but you need to call the reverse proxy as sub.domain.com. 192.168.0.123:80 won’t work, because the proxy has no idea which service you want to reach.

I found it really easy to set up with docker compose and caddy as a reverse proxy. Docker services on the same network automatically resolve their names so the configuration file for caddy (the reverse proxy) is literally just sub.mydomain.com { reverse_proxy jellyfin:12345 }. This will expose the jellyfin docker, which is listening on port 12345, as sub.mydomain.com on port 80.