It looks like it is downsampling the video or streaming after converting to another codec. Some codecs are fine for decoding on the server but the app may not support them so the server converts them. Some files are of higher quality than what the server is configured to deliver so it downsamples to stream it.

Check the configuration and look for anything to do with codecs, hardware decoding, streaming quality, and so on. It may also be on the app, so if you can access a different interface then test that to narrow down the issue.