What you may want to do is use something like Tailscale (or a VPN) to connect the two sites (homes) together.

Run your Tailscale setup in home A, and then in home B connected to your tailnet. You can install Tailscale on the home router of home B, or install it on any device that needs to connect to home A.

Then in home B you just connect to your Jellyfin server like you normally would if you were in home A. This could be a android box or smart tv for example or accessing it through a browser on a pc. (If those devices have Tailscale installed and are connected to your tailnet in home A)

Using a VPN like Tailscale would also mean you don’t expose your Jellyfin to the internet.