Have you isolated source formats? Does it do this with h264 and hevc?

Tried stopping other containers and/or all other services? Could have less to do with pure power and more with sheer volume (scheduling.)

Any issues if you use the current server as a file server and play the video on another machine? Any modifications to playback speed?

If none of that works you might try iperf3 to check network speeds. Or fio for disk speeds. Run tests covering all kinds of situations, you’re not so much looking for max speeds but instead for inconsistencies.

Don’t get too lost in the small stuff. Isolate systems. Might even move or clone the boot drive to another machine as a test. Try a different switch. That kind of thing.

Good luck!