I have no idea what to recommend for hardware but for OS I suggest Truenas Scale. It’s pretty easy to setup has a webui and it’s intuitive. As for hardware I say you go with a hba board for disks, a GPU for jellyfin transcoding and immich ai features, as for file sharing I use samba, which is kinda shitty but I didn’t found something easier.