Since you are against using plex even with using it only locally I would suggest you look into the Jellyfin clients support. Specifically look at each client device documentation and take a look at what audio and video codex they natively support. If there is one they both support then I suggest you do some testing and stream a movie or show that is encoded with that specific format. This will let you know if it will work across both clients whether it’s a stream issue hardware issue or Wi-Fi issue.

This will prevent transcoding that can cause audio sync issues.