Have you tried introducing unnecessary complexity?
Submitted 1 month ago by Ek-Hou-Van-Braai@piefed.social to selfhosted@lemmy.world
https://media.piefed.social/posts/Qw/sk/QwskwvFHcPlFovs.pn
Comments
truthfultemporarily@feddit.org 1 month ago
Sabata11792@ani.social 1 month ago
If you know how your setup works, then that’s a great time for another project that breaks everything.
cenzorrll@piefed.ca 1 month ago
Saturday morning: “Incus and podman seem interesting. I bet I could swap everything over while the family is out this afternoon”
Sunday evening: “Dad, when will the lights work again?”
InnerScientist@lemmy.world 1 month ago
Infrastructure diagram? No! In this homelab we refer to the infrastructure hyperdodecahedron.
tal@lemmy.today 1 month ago
It seems like a good time to learn graphviz’s dot format for the network layout diagrams, with automated layout.
non_burglar@lemmy.world 1 month ago
Haha too right mate
irmadlad@lemmy.world 1 month ago
unnecessary complexity?
I can help with that. It’s a skill I have. LOL
Zink@programming.dev 1 month ago
This is just as true in my non-computer hobbies that involve physical systems instead of code and configs!
If I had to just barely meet the requirements using as little budget as possible while making it easy for other people to work on, that would be called “work.” My brain needs to indulge in some over-engineering and “I need to see it for myself” kind of design decisions.
nonentity@sh.itjust.works 1 month ago
If it’s stable, it’s not a lab.
That’s infrastructure.
cenzorrll@piefed.ca 1 month ago
I’ve moved my homelab twice because it became stable, I really liked the services it was running, and I didn’t want to disturb the last lab*cough*prod server.
My current homelab will be moar containers. I’m sure I’ll push it to prod instead of changing the IP address and swapping name tags this time.
Prunebutt@slrpnk.net 1 month ago
When’s the last time you checked if your backup solution works?
Ek-Hou-Van-Braai@piefed.social 1 month ago
But if my backups actually work then I miss out on the joy of rebuilding everything from scratch and explaining to my wife why non of the lights in the house work anymore.
Tangent5280@lemmy.world 1 month ago
Carry around a candle in one of those old timey holders like Scrooge Mcduck
JetpackJackson@feddit.org 1 month ago
Yesterday! Switched my media server from freebsd to alpine and got the arr stack all set up using the backup zip files
halcyoncmdr@piefed.social 1 month ago
Backup? Psh… That’s what the lab is for.
piranhaconda@mander.xyz 1 month ago
What’s a backup solution…? (I’m only being half sarcastic, I really need to set one up, but it’s not as “fun” as the rest of my homelab, send suggestions)
tal@lemmy.today 1 month ago
logging is probably down
You do, of course have a dedicated rsyslogd server? An isolated system to which logs are sent, so that if someone compromises your other systems, they can’t wipe traces of that compromise from those systems?
Oh. You don’t. Well, that’s okay. Not every lab can be complete. That Raspberry Pi over there in the corner isn’t actually doing anything, but it’s probably happy where it is. You know, being off, not doing anything.
probable_possum@leminal.space 1 month ago
Ah. The approach that squirrel@piefed.zip suggested. ;)
Thanks for the tutorial though.
cenzorrll@piefed.ca 1 month ago
Hmmm. My pi{VPN,hole,dhcp,HA} has a little bit of overhead left…
Coleslaw4145@lemmy.world 1 month ago
No try migrating all your docker containers to podman.
fossilesque@mander.xyz 1 month ago
Don’t encourage me.
epicshepich@programming.dev 1 month ago
And then try turning on SELinux!
emerald@lemmy.blahaj.zone 1 month ago
And then migrate all your podman containers to proxmox
PHLAK@lemmy.world 1 month ago
Time to start documenting it!
irmadlad@lemmy.world 1 month ago
At 71, I have to document. I started a long time ago. I worked for a mec. contractor long ago, and the rule was: ‘If you didn’t write it down, it didn’t happen.’ That just carried over to everything I do.
Vile_port_aloo@lemmy.world 1 month ago
Do you write down what you write down on the internet?
BuboScandiacus@mander.xyz 1 month ago
NEVER1!!!11!!
sibannac@lemmy.world 1 month ago
Don’t look too closely you can jinx it.
tal@lemmy.today 1 month ago
You have remote power management set up for the systems in your homelab, right? A server set up that you can reach to power-cycle other servers, so that if they wedge in some unusable state and you can’t be physically there, you can still reboot them? A managed/smart PDU or something like that? Something like one of these guys?
Oh. You don’t. Well, that’s probably okay. I mean, nothing will probably go wrong and render a device in need of being forcibly rebooted when you’re physically away from home.
FauxLiving@lemmy.world 1 month ago
Oh. You don’t. Well, that’s probably okay. I mean, nothing will probably go wrong and render a device in need of being forcibly rebooted when you’re physically away from home.
*furiously adds a new item to the TODO list*
lemming741@lemmy.world 1 month ago
Does a $12 Shelly plug count?
TerHu@lemmy.dbzer0.com 1 month ago
if you can cycle your home assistant with the shelly plug whilst your home assistant is down, yes. from experience it’s really quite annoying to have a smart plug switch off HA…
Fmstrat@lemmy.world 1 month ago
I built an 8 outlet version of those with relays and wall outlets for… a lot less.
sytone@lemmy.world 1 month ago
Tal just got the chaotic evil tag today.
tychosmoose@lemmy.world 1 month ago
If you do have the smart PSU and power management server you probably also went down the rabbit hole of scripting the power cycling, right? Maybe made that server hardened against power loss disk corruption so it can be run until UPS battery exhaustion.
What if there is a power outage and NUT shuts everything down? Would be nice to have everything brought back up in an orderly way when power returns. Without manual intervention. But keeping you informed via logging and push notifications.
DownByLaw@sh.itjust.works 1 month ago
Have you already tried implementing an identity provider like Authentik, so you can add OIDC and ldap for all your services, while you are the only one that’s using them? 🤔
PumpkinEscobar@lemmy.world 1 month ago
Behind a traefik reverse proxy with lets encrypt for ssl even though the services aren’t exposed to the internet?
DownByLaw@sh.itjust.works 1 month ago
Don’t forget about Anubis and crowdsec to make it even safer inside your LAN
diablomnky666@lemmy.wtf 1 month ago
To be fair a lot of apps don’t handle custom CAs like they should. Looking at you Home Assistant! 😠
suicidaleggroll@lemmy.world 1 month ago
Who cares if it’s exposed to the internet?
-
Encrypting your local traffic is still valuable to protect your systems from any bad actors on your local network (neighbor kid cracks your wifi password, some device on your network decides to start snooping on your local traffic, etc)
-
Many services require HTTPS with a valid cert to function correctly, eg: Bitwarden. Having a real cert for a real domain is much simpler and easier to maintain than setting up your own CA
-
tal@lemmy.today 1 month ago
Probably a good idea to switch over to WPA-Enterprise using Authentik’s RADIUS server support and let all of the users of your wireless access point log in with their own network credentials, while you’re at it.
epicshepich@programming.dev 1 month ago
Hey my wife uses some of them too!
tal@lemmy.today 1 month ago
All of those systems in your homelab…they aren’t all pulling down their updates multiple times over your network link, right? You’re making use of a network-wide cache? For Debian-family systems, something like Apt-Cacher NG?
Oh. You’re not. Well, that’s probably okay. I mean, not everyone can have their environment optimized to minimize network traffic.
the_tab_key@lemmy.world 1 month ago
I set this up years ago, but then decided it was better to just install different distros on each of my computers. Problem solved?
panda_abyss@lemmy.ca 1 month ago
You can forgejo with a container index enabled, I don’t know if there’s a way to use that as a proxy for downloading containers though.
Abbysimons@lemmy.world 1 month ago
The rare moment when everything actually works. 😄
FauxLiving@lemmy.world 1 month ago
The comments in this thread have collectively created thousands of person-hours worth of work for us all…
MonkeMischief@lemmy.today 1 month ago
Don’t worry, you’re one Docker pull away from having to look up how to manually migrate Postgres databases within running containers!
(Looks at my PaperlessNGX container still down. Still irritated.)
damnthefilibuster@lemmy.world 1 month ago
Backups. You’re forgetting them.
paequ2@lemmy.today 1 month ago
Actually, one thing I want to do is switch from services being on a subdomain to services being on a path.
immich.myserver.com -> myserver.com/immich jellyfin.myserver.com -> myserver.com/jellyfin
I’m getting tired of having to update DNS records every time I want to add a new service.
I guess the tricky part will be making sure the services support this kind of routing…
Bakkoda@lemmy.world 1 month ago
I should do some breaking network changes… While tunneled in.
tal@lemmy.today 1 month ago
You have an intrusion detection system set up, right? A server watching your network’s traffic, looking for signs that systems on your network have been compromised, and to warn you?
Oh. You don’t. Well, that’s probably okay. I mean, probably nothing on your network has been compromised. And probably nothing in the future will be.
Avicenna@programming.dev 1 month ago
You can always configure your vim further
nucleative@lemmy.world 1 month ago
Never run:
docker compose down && docker compose up -d ``` Right before the end of your day. Ask me how I know 😂
Fedegenerate@fedinsfw.app 1 month ago
Going into spring/summer that’s ideal, I wanna go places do things. Mid winter, I’m feature creeping till something breaks.
tal@lemmy.today 1 month ago
All of your systems are set up, but are they capable of being redeployed using a configuration management software package? Ansible or something like that?
Oh. They’re not. Well, that’s probably okay. I mean, you could probably go manually reproduce configurations, more or less.
jeffep@lemmy.world 1 month ago
Can’t believe nobody here mentioned nixOS so far? How about moving all of your configs in a flake and manage all of your systems with it?
tal@lemmy.today 1 month ago
You have all your devices attached to a console server with a serial port console set up on the serial port, and if they support accessing the BIOS via a serial console, that enabled so that you can access that remotely, right? Either a dedicated hardware console server, or some server on your network with a multiport serial card or a USB to multiport serial adapter or something like that, right? So that if networking fails on one of those other devices, you can fire up
minicomor similar on the serial console server and get into the device and fix whatever’s broken?Oh, you don’t. Well, that’s probably okay. I mean, you probably won’t lose networking on those devices.
tal@lemmy.today 1 month ago
You have squid or some other forward http proxy set up to share a cache among all the devices on your network set up to access the Web, to minimize duplicate traffic?
And you have a shared caching DNS server set up locally, something like BIND?
Oh. You don’t. Well, that’s probably okay. I mean, it probably doesn’t matter that your devices are pulling duplicate copies of data down. Not everyone can have a network that minimizes latency and avoids inefficiency across devices.
Admax@lemmy.world 1 month ago
Then it turns out your monitoring system failed and FUCK IT’S BEEN A MONTH SINCE THE LAST PROPER BACKUP
fleem@piefed.zeromedia.vip 1 month ago
heck i really wish we could all throw a party together. part swap, stories swap. show off cool shit for everyone to copy.
help each other fill in the missing pieces
y’all seem like cool peeps meme-ing about shit nobody else gets!
time to test the backups!
AkatsukiLevi@lemmy.world 1 month ago
Do you have a spinning fish display in front of your homelab server, right? We all know the spinning fish improves performance and security, it is a indispensable part of homelabbing
Egonallanon@feddit.uk 1 month ago
Buy a UPS and setup a NUT server on the spare raspberry pi you have lying around.
Petter1@discuss.tchncs.de 1 month ago
You should use Arch, then you can update every 15 minutes 🤭
rumba@lemmy.zip 1 month ago
Nothing to install? Not with that attitude!
Start a 10" rack.
squirrel@piefed.zip 1 month ago
Let’s tinker around and accidentally break something.
wersooth@lemmy.ml 1 month ago
and debug it until you have to reinstall your entire stack from scarch
SpikesOtherDog@ani.social 1 month ago
GET OUT OF MY HOUSE!
rosco385@lemmy.wtf 1 month ago
Are you implying it’s possible to debug without having to reinstall from scratch? Preposterous! 😂
Bibip@programming.dev 1 month ago
Scarched arth
Dultas@lemmy.world 1 month ago
Guess this is a good time to test my infrastructure automation.
cenzorrll@piefed.ca 1 month ago
“Damn, I’ve got this Debian server shit down. I wonder how an opensuse server would work out” *installs tumbleweed *
True story
FreshLight@sh.itjust.works 1 month ago
My
manperson!