If everything is working correctly, whether or not you transcode is basically whether or not the end device can play the file without changes.

For example, my old Roku can play a raw 4K File under H264 with no problem. But if I throw an H265 at it, it requires the server to transcode. It also has problems with AAC audio. And my server is so old that just trying to rip the 4k apart entrance to the audio is often too much for it.

So to start, make sure your client device can play the files directly. If it can’t, you’re going to need to handbrake it before you put it on your server.