Submitted 6 days ago by LazerDickMcCheese@sh.itjust.works to selfhosted@lemmy.world
Great news! I started my selfhost journey over a year ago, and I’m finding myself needing better hardware. There’s so many services I want that my NAS can’t handle. And I unfortunately need to add GPU transcoding to my Jellyfin setup.
What’s the best OS for a machine focused on containers and (getting started with) VMs? I’ve heard Proxmox
What CPU specs should I be concerned about?
I’m willing to buy a pre-built as long as its hardware has sufficient longevity.
I think at this point I agree with the other commenter. If you’re strapped for storage it’s time to leave Synology behind, but it sounds more like it’s time to separate your app server from your storage server.
I use proxmox, and it was my primary when I got started with the same thing. I recommend build out storage in proxmox directly, that will be for VM images and container volumes. Then utilize regular backups to your Synology box. That way you have hot storage for drives and running things, cold storage for backups.
Then, inside your vms and containers you can mount things like media and other items from your Synology.
For you, I would recommend proxmox, then on top of that a big VM for running docker containers. In that VM you have all of your mounts from Synology into that VM, like Jellyfin stuff, and you pass those mounts into docker.
If you ever find yourself needing to stretch beyond the one box, then you can think about kubernetes or something, but I think that would be a good jump for now.
Why not use Proxmox to host the containers directly instead of using a VM? I know it’s easier to use this way but it kinda misses the point of using proxmox then
Not at all. Proxmox does a great job at hosting VMs and giving a control plane for them - but it does not do containers well. LXCs are a thing, and it hosts those - but never try to do docker in an LXC. (I tried so many different ways and guides and there were just too many caveats, and you end up always essentially giving root access to your containers, so it’s not great anyway). I’d like to see proxmox offer some sort of docker-first approach will it will manage volumes at the proxmox level, but they don’t seem concerned with that, and honestly if you’re doing that then you’re nearing kubernetes anyway.
Which is what I ended up doing - k3s on proxmox VMs. Proxmox handles the instances themselves, spins up a VM on each host to run k3s, and then I run k3s from within there. Same paradigm as the major cloud providers. GKE, AKS, and EKS all run k8s within a VM on their existing compute stack, so this fits right in.
That’s a great question…redundant ports and all that
Thanks, that’s some of the info I’m needing to make the jump over. How’s the learning curve? One of my big concerns is wrapping all of these things under Tailscale. It was easy on Synology, but Proxmox (I imagine) isn’t as straightforward. Eventually, I’d like to switch to headscale, but one thing at a time
Just focus on one project at a time, break it out into small victories that you can celebrate. A project like this is going to be more than a single weekend. Just get proxmox up and running. Then a simple VM. Then a backup job. Don’t try to get everything including tailscale working all at once. The learning curve is a bit more than you’re probably used to, but if you take it slow and focus on those small steps you’ll be fine.
Define goals. What services can’t be handled?
If transcoding is a goal build around intel. Quicksync video is a no brainer, imo. GPU is unnecessary power draw (15-25w+ idle depending on card) and waste of a pcie slot unless you want to do LLM stuff. Imo 10th gen intel is the sweet spot for quicksync unless you desperately need av1/vp9. If so then you need much more expensive 13/14 gen, which use more power and have more considerations for thermal management
OS is an endless debate. Proxmox is fine and free, why not try it? Unraid is easier to get your bearings but it does cost money. Debian is also free but a bit more confusing because not purpose built. Truenas as well. All can do containers and VMs, but approach in different ways. None is “best” but some are more “free” which is nice
CPU specs are dependent on goals. For transcoding as said above quicksync is necessary and is so impressive. I can transcode a 4k remux to one device while transcoding a 1080 remux to another and direct playing a 4k remux and cpu sits under 25% load on Xeon equivalent of 10700. You don’t need a Xeon btw, I just got a great deal where this was $50 (see next point). Otherwise specs depend wildly on what you plan to do. I can run windows VMs pretty well with this though for the handful of times I need a windows machine
Prebuilt is a waste. Used hardware is cheap and gives more options and can plan more. What are you willing to buy now and what do you eventually want? My NAS started as a 36tb array with 16gb ram and no cache, now it’s 234tb and 4tb cache with 32gb ecc ram years later. Slowly building up was easier on wallet and used hardware, refurb drives, etc is 100% the build. Your goals will likely vary but figure out your roadmap and go from there
Also keep in mind that not every service benefits from running on a NAS. My homeassistant server is run on a raspberry pi for example. Easier to keep it segregated and don’t have to worry about getting zwave/zigbee/mqtt/etc all working with a docker plus dealing with any server downtime impacting home. Tbf literally everything else is run on the nas though haha
In hindsight, I didn’t explain myself well enough. My plan is to use my current NAS as a NAS and little more; I’d like a machine with respectable hardware to handle what my NAS is currently running plus more.
My NAS has Jellyfin, arrs, all the stuff that goes with that, Pi-hole, and Homarr. And that’s pushing its limits: everything has been slow, streams freeze, I’ve had containers quit, etc.
I’d like to get into other projects like Radicale, Mealie, ErsatzTV (old PC could handle it, NAS can’t), CCTV, and more. But according to my resources, the NAS can’t handle it
GPU (for the sake of transcoding) isn’t worth it?
This makes sense
No sense in getting rid of hardware that is working. I’m not familiar with ersatztv but for all the other stuff I am able to handily run it on a 10th gen intel build that is also handling nas duties fwiw. And some stuff is not ideal (cctv is handled via blue iris, which runs in windows VM, everything else is docker)
for the gpu it really depends on your needs. How many users is the big one. If you have at most 2-4 concurrent users and that is an uncommon scenario the gpu is a waste of power, money, and thermal management. Igpu will sip power and transcode (depending on library content, again av1/vp9 on a 10th gen isn’t happening) with that user load assuming you have a decent amount of ram (I have 32gb so you don’t need absurd amounts).
However if you have a lot of users hitting you, 5-6+ or more concurrent streams that all transcode, then you need to start evaluating a discrete gpu (and maybe a significant internet connection bc damn). Alternatively you can suggest your users get something like a ugoos am6b+ flashed with coreelec or a similar setup that can just direct play basically anything but that’s a bit challenging to setup
My Synology is compatible with an expansion unit and can support two of them. Check if yours can do the same for the storage aspect.
Good looking out, I had no idea that was an option
Honestly I just run Alpine Linux on a mini PC (router) or Raspberry Pi (NAS). I don’t like to screw around with outdated, bloated Debian-based distros.
borari@lemmy.dbzer0.com 6 days ago
Depending on how many bays your Synology is, you might be best off getting a nuc or a mini pc for compute and using your synology just for storage.
LazerDickMcCheese@sh.itjust.works 6 days ago
It’s 4 bays, and we’re eating that space up quicker than I imagined
cRazi_man@europe.pub 2 days ago
I’m in the same boat. My 2 bay Synology is enough, but the apps and Docker containers are bringing it to its knees. I’m keeping the Synology and adding on a mini PC to run docker containers.
Found a mini PC with an i5-12500T for £240.
I’m a noob at this stuff, but I’m hyped to get more stability in what I host and start using more demanding services.
borari@lemmy.dbzer0.com 6 days ago
I have a 6 bay, so yeah that might be a little limiting. I have all my personal stuff backed up to an encrypted cloud mount, the bulk of my storage space is pirated media I could download again, and I have the Synology using SHR so I just plug in a bigger drive, expand the array, then plug in another bigger drive and repeat. Because of duplication sectors you might not benefit as much from that method with just 4 bays. Or if you have enough stuff you can’t feasible push to up to the cloud to give piece of mind during rebuilding I guess.
Bronzie@sh.itjust.works 5 days ago
I run a 4 bay and a N100 NUC.
The Synology is almost a pure storage machine. Works really well with Proxmox on the side. Not a single file has made it kneel yet, and I’ve thrown some high bitrate badboys on it.
Is not upgrading the drives an alternative?
I feel like you sacrifice a lot of practicality removing the NAS, such as automatic backup from phones and very easy remote access.
Personally I also prefer separating data and software, so I don’t lose it all if a component fails.
Just my .02
myersguy@lemmy.simpl.website 6 days ago
Plus one to this. It’s super nice separating concerns in this way.
curbstickle@lemmy.dbzer0.com 6 days ago
This is precisely what I do with my nas.
I have 9…ish tiny/mini/micros for compute, two NAS (locally).
Solid approach
Zikeji@programming.dev 6 days ago
9? That’s quite a bit of compute lol.
My journey started with 1 server, then 4, then 5 (one functioning as a NAS), then 1 (just the NAS box), then I moved and decided to slim it down to a proper NAS and 1 mini PC/NUC clone. Now I’m up to two because the first was an Intel N105 which just isn’t up for the challenges lol
Lem453@lemmy.ca 6 days ago
I did his when I moved from unraid because I wanted better infra as code for my dockers etc. Kept unraid with all my drives and use NFS mounts