Those who are hosting on bare metal: What is stopping you from using Containers or VM's? What are you self hosting?

Submitted ⁨⁨2⁩ ⁨months⁩ ago⁩ by ⁨kiol@lemmy.world⁩ to ⁨selfhosted@lemmy.world⁩

Curious to know what the experiences are for those who are sticking to bare metal. Would like to better understand who keeps such admins avoiding containers, Docker, Podman, Virtual Machines, etc. What keeps you on bare metal in 2025?

source

Comments

Sort:hotnew top

medem@lemmy.wtf ⁨2⁩ ⁨months⁩ ago
The fact that I bought all my machines used (and mostly on sale), and that not one of them is general purpose, id est, I bought each piece of hardware with a (more or less) concrete idea of what would be its use case. For example, my machine acting as a file server is way bigger and faster than my desktop, and I have a 20-year-old machine with very modest specs whose only purpose is being a dumb client for all the bigger servers. I develop programs in one machine and surf the internet and watch videos on the other. I have no use case for VMs besides the Logical Domains I setup in one of my SPARC hosts.

source
lka1988@lemmy.dbzer0.com ⁨2⁩ ⁨months⁩ ago
I run my NAS and Home Assistant on bare metal.

NAS: OMV on a Mac mini with a separate drive case

Home Assistant: HAOS on a Lenovo M710q, since 1) it has a USB zigbee adapter and 2) HAOS on bare metal is more flexible

Both of those are much easier to manage on bare metal. Everything else runs virtualized on my Proxmox cluster, whether it’s Docker stacks on a dedicated VM, an application that I want to run separately in an LXC, or something heavier in its own VM.
source
misterbngo@awful.systems ⁨2⁩ ⁨months⁩ ago
Your phrasing of the question implies a poor understanding. There’s nothing preventing you from running containers on bare metal.

My colo setup is a mix of classical and podman systemd units running on bare metal, combined with a little nginx for the domain and tls termination.

I think you’re actually asking why folks would use bare metal instead of cloud and here’s the truth. You’re paying for that resiliency even if you don’t need it which means that renting the cloud stuff is incredibly expensive. Most people can probably get away with a$10 vps, but the aws meme of needing 5 app servers, an rds and a load balancer to run WordPress has rotted people. My server that I paid a few grand for on eBay would cost me about as much monthly to rent from aws. I’ve stuffed it full of flash with enough redundancy to lose half of it before going into colo for replacement. I paid a bit upfront but I am set on capacity for another half decade plus, my costs are otherwise fixed.

source
- lazynooblet@lazysoci.al ⁨2⁩ ⁨months⁩ ago
  
  Your phrasing of the question implies poor understanding.
  
  Your phrasing of the answer implies poor understanding. The question was why bare metal vs containers/VMs.
  
  source
  - sepi@piefed.social ⁨1⁩ ⁨month⁩ ago
    The phrasing by the person you are responding to is perfectly fine and shows ample understanding. Maybe you do not understand what they were positing.
    
    source
eleitl@lemmy.zip ⁨2⁩ ⁨months⁩ ago
Obviously, you host your own hypervisor on own or rented bare metal.

source
hperrin@lemmy.ca ⁨2⁩ ⁨months⁩ ago
There’s one thing I’m hosting on bare metal, a WebDAV server. I’m running it on the host because it uses PAM for authentication, and that doesn’t work in a container.

source
Kurious84@lemmings.world ⁨2⁩ ⁨months⁩ ago
Anything you want dedicated performance on or require fine tuning for a specific performance use cases. Theyre out there.

source
Surp@lemmy.world ⁨2⁩ ⁨months⁩ ago
What are you doing running your vms on bare metal? Time is a flat circle.

source
- missfrizzle@discuss.tchncs.de ⁨2⁩ ⁨months⁩ ago
  for work I have a cloud dev VM, in which I run WSL2. so there’s at least two levels of VMs happening, maybe three honestly.
  
  source
brucethemoose@lemmy.world ⁨2⁩ ⁨months⁩ ago
In my case it’s performance and sheer RAM usage.

GLM 4.5 needs like 112GB RAM and absolutely every megabyte of VRAM from the GPU. It simply cannot afford the overhead. I think containers may slow down CPU<->GPU transfers slightly, but don’t quote me on that.

source
- kiol@lemmy.world ⁨2⁩ ⁨months⁩ ago
  Can anyone confirm if containers would actually impact CPU to GPU transfers
  
  source
  - brucethemoose@lemmy.world ⁨2⁩ ⁨months⁩ ago
    To be clear, VMs absolutely have overhead but Docker/Podman is the question. It might be negligible.
    
    And this is a particularly weird scenario (since prompt processing literally has to shuffle 112GB over the PCIe bus for each batch). Most GPGPU apps aren’t so sensitive to that.
    
    source
corsicanguppy@lemmy.ca ⁨2⁩ ⁨months⁩ ago
I don’t host on containers because I used to do OS security for a while.

source
- kiol@lemmy.world ⁨2⁩ ⁨months⁩ ago
  Say more, what did that experience teach you? And, what would you do instead?
  
  source
Andres4NY@social.ridetrans.it ⁨2⁩ ⁨months⁩ ago
@kiol I mean, I use both. If something has a Debian package and is well-maintained, I'll happily use that. For example, prosody is packaged nicely, there's no need for a container there. I also don't want to upgrade to the latest version all the time. Or Dovecot, which just had a nasty cache bug in the latest version that allows people to view other peoples' mailboxes. Since I'm still on Debian 12 on my mail server, I remain unaffected and I can let the bugs be shaken out before I upgrade.
source
- Andres4NY@social.ridetrans.it ⁨2⁩ ⁨months⁩ ago
  @kiol On the other hand, for doing builds (debian packages and random other stuff), I'll use podman containers. I've got a self-built build environment that I trust (debootstrap'd), and it's pretty simple to create a new build env container for some package, and wipe it when it gets too messy over time and create a new one. And for building larger packages I've got ccache, which doesn't get wiped by each different build; I've got multiple chromium build containers w/ ccache, llvm build env, etc
  source
  - Andres4NY@social.ridetrans.it ⁨2⁩ ⁨months⁩ ago
    @kiol And then there's the stuff that's not packaged in Debian, like navidrome. I use a container for that for simplicity, and because if it breaks it's not a big deal - temporary downtime of email is bad, temporary downtime of my streaming flac server means I just re-listen to the stuff that my subsonic clients have cached locally.
    source
    -> View More Comments
akincisor@sh.itjust.works ⁨2⁩ ⁨months⁩ ago
I have a single micro itx htpc/media server/nas in my bedroom. Why use containers?

source
- kiol@lemmy.world ⁨2⁩ ⁨months⁩ ago
  Do you back it up?
  
  source
  - akincisor@sh.itjust.works ⁨2⁩ ⁨months⁩ ago
    My raid is rsync :)
    
    source
towerful@programming.dev ⁨2⁩ ⁨months⁩ ago
I would always run proxmox to set up docker VMs.

I found Talos Linux, which is a dedicated distro for kubernetes. Which aligned with my desire to learn k8s.
It was great. I ran it as bare-metal on a 3 node cluster. I learned a lot, I got my project complete, everything went fine.
I will use Talos Linux again.
However next time, I’m running proxmox with 2 VMs per node - 3 talos control VMs and 3 talos worker VMs.
I imagine running 6 servers with Talos is the way to go. Running them hyperconverged was a massive pain. Separating control plane and data/worker plane (or whatever it is) makes sense - it’s the way k8s is designed.
It wasn’t the hardware that had issues, but various workloads. And being able to restart or wipe a control node or a worker node would’ve made things so much easier.

Also, why wouldn’t I run proxmox?
Overhead is minimal, get nice overview, get a nice UI, and I get snapshots and backups

source
- possiblylinux127@lemmy.zip ⁨2⁩ ⁨months⁩ ago
  What hardware are you running it on?
  
  source
  - towerful@programming.dev ⁨2⁩ ⁨months⁩ ago
    3x minisforums MS-01
    
    source
9tr6gyp3@lemmy.world ⁨2⁩ ⁨months⁩ ago
I thought about running something like proxmox, but everything is too pooled, too specialized, or proxmox doesn’t provide the packages I want to use.

Just went with arch as the host OS and firejail or lxc any processes i want contained.

source
- towerful@programming.dev ⁨2⁩ ⁨months⁩ ago
  I’ve never installed a package on proxmox.
  I’ve BARELY interacted with CLI on proxmox (I have a script that creates a nice Debian VM template, and occasionally having to really kill a VM).
  
  What would you install on proxmox?!
  
  source
  - 9tr6gyp3@lemmy.world ⁨2⁩ ⁨months⁩ ago
    Firmware update utilities, host OS file system encryption packages, HBA management tools, temperature monitoring, and then a lot of the packages had bugs that were resolved with newer versions, but proxmox only provided old versions.
    
    source
    -> View More Comments
LifeInMultipleChoice@lemmy.world ⁨2⁩ ⁨months⁩ ago
For me it’s lack of understanding usually. I haven’t sat down and really learned what docker is/does. And when I tried to use it once I ended up with errors (thankfully they all seemed contained by the docker) but I just haven’t gotten around to looking more into than seeing suggestions to install say Pihole in it. Pretty sure I installed Pihole outside of one. Jellyfin outside, copyparty outside, and I something else im forgetting at the moment.

I was thinking of installing a chat app in one, but I put off that project because I got busy at work and it’s not something I normally use.

I guess I just haven’t been forced to see the upsides yet. But am always wanting to learn

source
- slazer2au@lemmy.world ⁨2⁩ ⁨months⁩ ago
  containerisation is to applications as virtual machines are to hardware.
  
  VMs share the same CPU, memory, and storage on the same host.
  Containers share the same binaries in an OS.
  
  source
  - LifeInMultipleChoice@lemmy.world ⁨2⁩ ⁨months⁩ ago
    When you say binaries do you mean locally stored directories kind of like what Lutris or Steam would do for a Windows game. (Create a false c:\ )
    
    source
    -> View More Comments
tofu@lemmy.nocturnal.garden ⁨2⁩ ⁨months⁩ ago
TrueNAS is on bare metal has I have a dedicated NAS machine that’s not doing everything else and also is not recommended to virtualize. Not sure if that counts.

Same for the firewall (opnsense) since it is it’s own machine.

source
- kiol@lemmy.world ⁨2⁩ ⁨months⁩ ago
  Have you tried running containers on Truenas?
  
  source
  - tofu@lemmy.nocturnal.garden ⁨2⁩ ⁨months⁩ ago
    No because I run my containers elsewhere, not on the NAS
    
    source
DarkMetatron@feddit.org ⁨2⁩ ⁨months⁩ ago
My servers and NAS were created long before Docker was a thing, and as I am running them on a rolling release distribution there never was a reason to change anything. It works perfectly fine the way it is, and it will most likely run perfectly fine the next 10+ years too.

Well I am planning, when I find the time to research a good successor, to replace my aging HPE ProLiant MicroServer Gen8 that I use as Homeserver/NAS. Maybe I will then setup everything clean and migrate the services to docker/podman/whatever is fancy then. But most likely I will only transfer all the disks and keep the old system running on newer hardware. Life is short…

source
ieGod@lemmy.zip ⁨2⁩ ⁨months⁩ ago
You sure you mean bare metal here? Bare metal means no OS.

source
kossa@feddit.org ⁨2⁩ ⁨months⁩ ago
Well, that is how I started out. Docker was not around yet (or not mainstream enough, maybe). So it is basically a legacy thing.

My main machine is a Frankenstein monster by now, so I am gradually moving. But since the days when I started out, time has become a scarce resource, so the process is painfully slow.

source
pineapplelover@lemmy.dbzer0.com ⁨2⁩ ⁨months⁩ ago
All I have is Minecraft and a discord bot so I don’t think it justifies vms

source
SailorFuzz@lemmy.world ⁨2⁩ ⁨months⁩ ago
Mainly that I don’t understand how to use containers… or VMs that well… I have and old MyCloud NAS and little pucky PC that I wanted to run simple QoL services on… HomeAssistant, JellyFin etc…

I got Proxmox installed on it, I can access it… I don’t know what the fuck I’m doing… There was a website that allowed you to just run scripts on shell to install a lot of things… but now none of those work becuase it says my version of Proxmox is wrong (when it’s not?)… so those don’t work…

And at least VMs are easy(ish) to understand. Fake computer with OS… easy. I’ve built PCs before, I get it… Containers just never want to work, or I don’t understand wtf to do to make them work.

I wanted to run a Zulip or Rocket.chat for internal messaging around the house (wife and I both work at home, kid does home/virtualschool)… wanted to use a container because a service that simple doesn’t feel like it needs a whole VM… but it won’t work…

source
- ChapulinColorado@lemmy.world ⁨2⁩ ⁨months⁩ ago
  I would give docker compose a try instead. I found Proxmox to be too much, when a simple yaml file (that can be checked into a repo) can do the job.
  
  Pay attention to when people say things can be improved (secrets/passwords, rootless/podman, backups), etc. And come back to them later.
  
  Just don’t expose things to the internet until you understand the risks and don’t check in secrets to a public git repo and go from there. It is a lot more manageable and feels like a hobby vs feeling like I’m still at work trying to get high availability, concurrency and all this other stuff that does not matter for a home setup.
  
  source
  - lka1988@lemmy.dbzer0.com ⁨2⁩ ⁨months⁩ ago
    
    I would give docker compose a try instead. I found Proxmox to be too much, when a simple yaml file (that can be checked into a repo) can do the job.
    
    Proxmox and Docker serve two completely different purposes.
    
    source
    -> View More Comments
Routhinator@startrek.website ⁨2⁩ ⁨months⁩ ago
I’m running Kube on baremetal.

source
frezik@lemmy.blahaj.zone ⁨2⁩ ⁨months⁩ ago
My file server is also the container/VM host. It does NAS duties while containers/VMs do the other services.

OPNsense is its own box because I prefer to separate it for security reasons.

Pihole is on its own RPi because that was easier to setup. I might move that functionality to the AdGuard plugin on OPNsense.

source
- HiTekRedNek@lemmy.world ⁨2⁩ ⁨months⁩ ago
  My reasons for keeping OpnSense on bare metal mirror yours. But additionally I don’t want my network to take a crap because my proxmox box goes down.
  
  I constantly am tweaking that machine…
  
  source
7rokhym@lemmy.ca ⁨2⁩ ⁨months⁩ ago
Not knowing about Incus (LXD). It’s a life changer. Would never run any service on bare metal again.

Using GenAI to develop my Terraform and Ansible playbooks is magical. Also, use it to document everything in beautiful HTML docs from the outputs. Amazing.

source
jaemo@sh.itjust.works ⁨2⁩ ⁨months⁩ ago
I generally abstract to docker anything I don’t want to bother with and just have it work.

If I’m working on something that requires lots of back and forth syncing between host and container, I’ll run that on bare metal and have it talk to things in docker.

Ie: working on an app or a website or something in language of choice on framework of choice, but postgres and redis are living in docker. Just the app I’m messing with and it’s direct dependencies run outside.

source
raman_klogius@ani.social ⁨2⁩ ⁨months⁩ ago
[deleted]
source
- kiol@lemmy.world ⁨2⁩ ⁨months⁩ ago
  I mean, I did
  
  source
  - raman_klogius@ani.social ⁨1⁩ ⁨month⁩ ago
    did what? apart from downvoting?
    
    source
TheMightyCat@ani.social ⁨2⁩ ⁨months⁩ ago
I’m selfhosting Forgejo and i don’t really see the benefit of migrating to a container, i can easily install and update it via the package manager so what benefit does containerization give?

source
- turmoil@feddit.org ⁨2⁩ ⁨months⁩ ago
  If you don’t exceed a single deployment, it’s fine.
  
  If however in the future, you want to add additional services to your host, let’s say an alerting or status system, it’s a lot easier to declare everything in a single place and then attach a reverse proxy to manage networking multiple services on one host.
  
  source
Jerry@feddit.online ⁨2⁩ ⁨months⁩ ago
Depends on the application for me. For Mastodon, I want to allow 12K character posts, more than 4 poll question choices, and custom themes. Can’t do it with Docker containers. For Peertube, Mobilizon, and Peertube, I use Docker containers.

source
- kiol@lemmy.world ⁨2⁩ ⁨months⁩ ago
  Why could you not have that Mastodon setup in containers? Sounds normal afaik
  
  source
  - farcaller@fstab.sh ⁨2⁩ ⁨months⁩ ago
    I’ll chime in: simplicity. It’s much easier to keep a few patches that apply to local OS builds: I use Nix, so my Mastodon microVM config just has an extra patch line. If there’s a new Mastodon update, the patch most probably will work for it too.
    
    Yes, I could build my own Docker container, but you can’t easily build it with a patch (for Mastodon specifically, you need to patch js pre-minification). It’s doable, but it’s quite annoying. And then you need to keep track of upstream and update your Dockerfile with new versions.
    
    source
bizarroland@lemmy.world ⁨2⁩ ⁨months⁩ ago
I’m running a TrueNAS server on bare metal with a handful of hard drives. I have virtualized it in the past, but meh, I’m also using TrueNAS’s internal features to host a jellyfin server and a couple of other easy to deploy containers.

source
- kiol@lemmy.world ⁨2⁩ ⁨months⁩ ago
  So Truenas itself is running your containers?
  
  source
  - bizarroland@lemmy.world ⁨2⁩ ⁨months⁩ ago
    Yeah, the more recent versions basically have a form of Docker as part of its setup.
    
    I believe it’s now running on Debian instead of free BSD, which probably simplified the containers set up.
    
    source
tychosmoose@lemmy.world ⁨2⁩ ⁨months⁩ ago
I’m doing this on a couple of machines. Only running NFS, Plex (looking at a Jellyfin migration soon), Home Assistant, LibreNMS and some really small other stuff. Not using VMs or LXC due to low-end hardware (pi and older tiny pc). Not using containers due to lack of experience with it and a little discomfort with the central daemon model of Docker, running containers built by people I don’t know.

The migration path I’m working on for myself is changing to Podman quadlets for rootless, more isolation between containers, and the benefits of management and updates via Systemd. So far my testing for that migration has been slow due to other projects. I’ll probably get it rolling on Debian 13 soon.

source
51dusty@lemmy.world ⁨2⁩ ⁨months⁩ ago
my two bare metal servers are the file server and music server. I have other services in a pi cluster.

file server because I can’t think of why I would need to use a container.

the music software is proprietary and requires additional complications to get it to work properly…or at all, in a container. it also does not like sharing resources and is CPU heavy when playing to multiple sources.

if either of these machines die, a temporary replacement can be sourced very easily(e.g. the back of my server closet) and recreated from backups while I purchase new or fix/rebuild the broken one.

IMO the only reliable method for containers is a cluster because if you’re running several containers on a device and it fails you’ve lost several services.

source
- kiol@lemmy.world ⁨2⁩ ⁨months⁩ ago
  Cool, care to share more specifics on your Pi Cluster
  
  source
  - 51dusty@lemmy.world ⁨2⁩ ⁨months⁩ ago
    I followed one of the many guides for installing proxmox on Rpis. 3node, 4gb rpi4s
    
    I use the cluster for lighter services like Trilium, FreshRss, secondary DNS, a jumpbox… and something else I forget. I’m going to try immich and see how it performs.
    
    my recent goto for cheap($200-300) servers are Debian + old Intel Macbook pros. I have two Minecraft bedrock servers on MBPs… one an i5, the other an i7.
    
    I also use a Lenovo laptop to host some industrial control software for work.
    
    source

-> View More Comments