Maybe have a look at https://nginxproxymanager.com as well. I don't know how difficult it is to install since I never used it, but I heard it has a relatively straight-forward graphical interface.

Configuring good old plain nginx isn't super complicated. It depends a bit on your specific setup, though. Generally, you'd put config files into /etc/nginx/sites-available/servicexyz (or put it in the default)

server {
    listen 80;
    server_name jellyfin.yourdomain.com;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl;
    server_name jellyfin.yourdomain.com;

    ssl_certificate /etc/ssl/certs/your_ssl_certificate.crt;
    ssl_certificate_key /etc/ssl/private/your_private_key.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;

    location / {
        proxy_pass http://127.0.0.1:8096;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

    access_log /var/log/nginx/jellyfin.yourdomain_access.log;
    error_log /var/log/nginx/jellyfin.yourdomain_error.log;
}

It's a bit tricky to search for tutorials these days... I got that from: https://linuxconfig.org/setting-up-nginx-reverse-proxy-server-on-debian-linux

Jellyfin would then take all requests addressed at jellyfin.yourdomain.com and forward that to your Jellyfin which hopefully runs on port 8096. You'd use a similar file like this for each service, just adapt them to the internal port and domain.

You can also have all of this on a single domain (and not sub-domains). That'd be the difference between "jellyfin.yourdomain.com" and "yourdomain.com/jellyfin". That's accomplished with one file with a single "server" block in it, but make it several "location" blocks within, like location /jellyfin

Alright, now that I wrote it down, it certainly requires some knowledge. If that's too much and all the other people here recommend Caddy, maybe have a look at that as well. It seems to be packaged in Debian, too.