I’ve had similar requirements for a few years. Finally settled on 8 bay synology with a dell micro optiplex. I have a 10gb card in the synology and actually use a usb to 2.5gbE in the dell with zero stability issues. Some synology products can transcode a stream or two of 4k with quick sync but not many of the 8 bay ones. I have a 9th gen i5 with 32gb of ram that handles all my transcoding needs. Done 2x 4k transcodes with no issues while also streaming other 1080p for additional users.