Comment on Holiday Upgrade Disasters
Onomatopoeia@lemmy.cafe 15 hours ago
I don’t do upgrades (well, not in the sense most people think of them).
My approach is that upgrades are too risky, things always break. It’s also why I don’t permit auto updates on anything. I’d rather do manual updates than dedicated time. Keeping things working is more important, and I have backups.
I run everything virtualized (as much as I can), so I can test upgrades by cloning a system and upgrading the clone. If that fails, I simply build a new system based on some templates I keep. Run in parallel, copy config and data as best I can, then migrate.
This is a common methodology in enterprise, which virtualization makes a lot easier for us self hosters.
That’s an interesting process… you could improve it with some ansible - if that’s your thing… or use snapshots on the VM(s) and roll back?
That’s kinda what I’m doing with this (physical obviously) Pi… take a full backup now and again… do upgrades… rollback when completely borked.
Brkdncr@lemmy.world 14 hours ago
It’s not common in enterprise to not auto-update.
frongt@lemmy.zip 12 hours ago
Depends on the company and the system. Some of them need to be done off-hours while people aren’t using them. Some are HA and/or insignificant enough that you can do them any time without interruption.
femtek@lemmy.blahaj.zone 12 hours ago
I have not had any auto update in my entire career but i have always been on the infra side so it was never just my app. Though I do wish the update/outage process would be updated to reflect kubernetes and not just think of everything as monolithic servers.
ExcessShiv@lemmy.dbzer0.com 10 hours ago
It’s extremely common…most production lines I’ve ever been to only do manual updates on equipment, if any at all.
suzune@ani.social 12 hours ago
It’s also not uncommon in enterprises that things break needlessly.