How do you document your Homelab?

Submitted ⁨⁨5⁩ ⁨weeks⁩ ago⁩ by ⁨MIXEDUNIVERS@discuss.tchncs.de⁩ to ⁨selfhosted@lemmy.world⁩

Hello, how do you document your home lab? Whether it’s a small server or a big one with firewall and more nodes. I have a small pc with Proxmox and there I have a VM with OpnSense. After I’ve entered my VPN as a interface in OpenSense, I noticed that I slowly lose the overview with the different rules that I have built in my firewall. And I know that my setup is relatively easy in comparison to others here in this community. I want to have a quick Overview at the various VMs, like the Lxc container, Docker containers that I have in this and the IP addresses that I have assigned to them. I search for a simple an intuitiv way for beginners.

source

Comments

Sort:hotnew top

justcallmelarry@lemmy.dbzer0.com ⁨5⁩ ⁨weeks⁩ ago
Image

Jokes aside, I do keep some harder to remember stuff written down in a README.md in my repo, but mainly most things are undocumented

source
- foggy@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
  I operate on the philosophy that it is better for me to relearn things that lean on old documentation that may no longer be accurate/relevant.
  
  The best way to implement a safe connection to my home lab today might not be the safest way tomorrow.
  
  Old dog, new tricks, etc.
  
  Also! Your documentation is an attackers wet dream.
  
  source
  - negativenull@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
    Security by obscurity, baby!
    
    source
    -> View More Comments
  - TrickDacy@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
    I’m gonna try this neat trick at work
    
    source
  - CapitalNumbers@lemm.ee ⁨5⁩ ⁨weeks⁩ ago
    while security might be compromised if an attacker found your documentation, it could equally be compromised by having zero documentation
    
    the easier it is for you to get things back up and running in the event of a data loss / corrupted hard drive / new machine / etc, the less likely you are to forget any crucial steps (eg setting up iptables or ufw)
    
    source
    -> View More Comments
  - Unforeseen@sh.itjust.works ⁨5⁩ ⁨weeks⁩ ago
    I do this continually for work as well, I approach every new project assuming best practice or approach options have changed. It doesn’t matter how experienced I am in what I’m doing, I still loop back and check.
    
    It’s such an automatic thing I don’t even think about it, but honestly not sure if it’s because of interest or because of fear of being called out for doing something wrong lol
    
    source
- jagermo@feddit.org ⁨5⁩ ⁨weeks⁩ ago
  “Shit, i hope i remeber the key words i searched for”
  
  source
- nfreak@lemmy.ml ⁨5⁩ ⁨weeks⁩ ago
  My wife was mentioning the other day that if something happened to me she’d have absolutely no idea how to work any of this shit and that convinced me to actually start documenting it LMAO
  
  Good time to start doing it too. Aside from setting up a NAS this weekend, I’ve got this project right where I want it for a long while.
  
  source
- redlemace@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
  Guilty too. There are names on router- and switch interfaces. Servers get fixed IP from dhcp so is in the note field there too. That’s about it
  
  source
nagaram@startrek.website ⁨5⁩ ⁨weeks⁩ ago
I download the YouTube tutorials I followed, upload them to my UAT Jellyfin server, and then when my server is having issues I can’t get to the videos!

A flawless system really

source
tauren@lemm.ee ⁨5⁩ ⁨weeks⁩ ago
Lol.

source
loudWaterEnjoyer@lemmy.dbzer0.com ⁨5⁩ ⁨weeks⁩ ago
I am fortunate enough to only manage a homelab and not an enterprise sized network. So I don’t document anything just like at work.

source
- whereisk@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
  Brill. One of us.
  
  source
atzanteol@sh.itjust.works ⁨5⁩ ⁨weeks⁩ ago
I build my infrastructure with the terraform, Ansible and helm charts. The code is it’s own documentation as well as comments in that code explaining why I’ve done things if it’s not obvious.

source
- ch8zer@lemmy.ca ⁨5⁩ ⁨weeks⁩ ago
  This really is the way.
  
  It goes beyond documentation too - it allows me to migrate to new hosts or to easily automate upgrading the OS release version.
  
  I have a docusaurus site for my homeland and I have ansible and terraform generate files for the docs so I don’t have to record anything. Some of the stuff I note down:
  
  DNS leases
  
  General infra diagrams
  
  IP info
  
  Host info
  
  source
dabe@lemmy.zip ⁨5⁩ ⁨weeks⁩ ago
95% of my homelab lives on a single server, and everything I do is within containers. So, my documentation is just keeping all my compose files in a git repo and writing in comments when necessary. It’s fairly self-documenting, and I haven’t found the need to break out of just using containers for everything, besides a couple things like setting up mergerfs or cockpit, but that’s all plug and play nowadays with stuff like projectucore.io

Of course, I don’t have any other things set up in my physical layout or network stack… but all that stuff would probably just go into an entry in my notes (obsidian/wiki.vim).

source
voklen@programming.dev ⁨5⁩ ⁨weeks⁩ ago
I just have NixOS on mine and I write everything in comments in my configuration.nix. Usually I’ll write notes about why I set a certain option above the line where I do it but I also have a section at the start that describes anything to be aware of if I’m moving this configuration to another machine. For example “This assumes a drive is mounted on /mnt/backup0”

source
brayd@discuss.tchncs.de ⁨5⁩ ⁨weeks⁩ ago
I just created a note in Obsidian with the Excalidraw plugin.

source
- neatobuilds@lemmy.today ⁨5⁩ ⁨weeks⁩ ago
  I use obsidian and ezcalidraw also, I didn’t realize we can put one in the other
  
  source
rimjob_rainer@discuss.tchncs.de ⁨5⁩ ⁨weeks⁩ ago
I don’t, and I know that I’m fucked one day.

source
MangoPenguin@lemmy.blahaj.zone ⁨5⁩ ⁨weeks⁩ ago
I generally just make notes in Obsidian, mostly about switch ports, VLANs, IP assignments and that kind of thing.

source
Cyber@feddit.uk ⁨5⁩ ⁨weeks⁩ ago
A combination of Logseq (what, why, hiw) and KeePass for IPs and passwords (obviously)… I use the heirarchy in Keepass to show a device and then the services on it and then their configs, ie

Hypervisor1

VM1

root user details

VM2 (Etc)

I used to do Visio drawings, but they were always out of date.
source
exu@feditown.com ⁨5⁩ ⁨weeks⁩ ago
It’s really a wild growth over the years. My current approach is twofold. Netbox to manage devices/VMs and associated info with service deployments using Ansible. You can use the info from Netbox as an Ansible inventory directly.

Previously I tried network diagrams (too low detail) and spreadsheets (terrible to modify) to document machines. And for serviced I’d have an install page on my wiki (apologies, the codeblocks are somewhat broken atm)

source
- suzune@ani.social ⁨5⁩ ⁨weeks⁩ ago
  I run Netbox for documentation only. But your approach is what I actually wanted to do, if I just had plenty of time.
  
  source
  - pezhore@infosec.pub ⁨5⁩ ⁨weeks⁩ ago
    I use netbox too - and if you’re careful about it, you can actually use terraform to create the netbox details. I use one manifest file to handle deployment to Proxmox, set up DNS in PowerDNS, and create the relevant netbox entries.
    
    source
- irmadlad@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
  The way you go about it on your wiki, is almost the same process/format, tho not as fancy . I’ll even throw in a couple links to tuts I found useful for that particular segment in the notes.
  
  source
thirdBreakfast@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
I have it in a git repo, broken down by the nodes and vps names. In each of these folders is a mixture of Ansible playbooks, docker compose or just markdown files with the descriptions. Some is random stuff - my VPS allows the export of the cloud firewalls as JSON for instance. All the secrets needed by Ansible are in an Ansible vault, the rest in KeePass.

source
ryanpdg1@lemmy.ca ⁨5⁩ ⁨weeks⁩ ago
I use the notes sections in proxmox preeettty heavily. Lots of links to the helper scripts, youtube videos and other resources i used to get er’ goin’.

In the near future I’m really hoping I can set up Netbox to help me document the network and equipment I’m putting in my homelab. a nice thing is that I went through a divorce a while ago and I’m getting to start from scratch. You’d be surprised at just how much you’ve learned since starting to self host and I think there’s this sunk cost fallacy that gets a lot of us to keep going with what we’ve got already set up because we’ve “already put so much work into it” and the concern of what we might lose by scrapping it and starting over.

Also, not what you asked… but if you’re still relatively new with proxmox you should check out the ProxmoxVE helper scripts. Lots of good automated scripts from doing a post-install to setting up various LXC containers and VM’s

source
- SidewaysHighways@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
  RIP Tteck! (not sure if i spelled it right)
  
  source
413j0@lemmy.dbzer0.com ⁨5⁩ ⁨weeks⁩ ago
I just set up everything via ansible and comment everything religiously, I’ve gotten to the point where I have playbooks for my laptop and desktop so if I have do do a clean install I just have to run the playbook and everything is set up and installed to my liking

It’s annoying to go to my playbook to make changes and rerun it every time I want to install or remove anything from my daily PCs, but it pays off when I migrate computers or for some reason have to do a clean install

And for high level stuff and things not in ansible I keep a tiddlywiki since no matter how catastrophic the failure of my systems I can always find a way to access an HTML file for my own reference

source
cecilkorik@lemmy.ca ⁨5⁩ ⁨weeks⁩ ago
I’ve moved to an “infrastructure as code” approach, not using any fancy tools in particular, primarily just bash shell scripts. Basically almost everything I setup or do gets documented via shell scripts, I write them as I go when I’m learning to install something new, and before I commit to something to new, I take extra care to make sure the scripts are idempotent so that when I want to do make any changes, all I need to do is add it to the appropriate script and re-run it.

The idempotent part takes some effort sometimes, but is not actually as hard as it seems, particularly if you don’t mind that it sometimes spends some wasted time doing things that have already been done, and occasionally spits out some harmless error messages because something is already done, but I also try to minimize that when I can. The consequences of doing too much by re-running are rarely serious. Yeah sometimes the scripts can break, but as long as they fail properly (set -euo pipefail) it’s usually pretty obvious how to fix it and it won’t leave too much of a mess.

Doing this has transformed my homelab from a mess of unknowable higgledy-piggledy spaghetti-services that was always teetering one small failure away from total collapse and frantic rebuilding, into something repeatable and reproducible that I can actually … wait for it … test. Just firing up a Linux ISO in a VM is all I need to test everything I’m doing in a perfect sandbox, and I can throw it away when I’m done with no regrets. Plus it makes rolling out new servers, and more importantly, decommissioning old ones, a breeze, you know exactly what’s on them and how it was set up, because it was all in your scripts. Combined with good data backups (which are also set up in the scripts) and restores (which I also test with scripts)

Yeah there are probably easier ways to accomplish what I’m doing using some of the technologies like terraform, ansible and nix/flake that people have mentioned, and I’ve dabbled with those, but for me, the shell script approach strikes a nice balance of not just documenting but also learning the process myself so that I understand enough of what it’s doing to effectively debug it when something goes wrong, and it works on almost everything and in most cases requires no installation or setup. Bash is everywhere. I even have an infrastructure-as-code setup for my Steam Deck to install stuff and get it set up the way I want.

source
- CapitalNumbers@lemm.ee ⁨5⁩ ⁨weeks⁩ ago
  i second this
  
  i haven’t gotten around to looking into something like terraform/ansible yet, and currently rely on a series of setup.sh scripts and docker-compose files
  
  i have a single master setup.sh at the root of my homelab which basically just outlines which scripts i need to run and in what order in order to get things back up and running from zero
  
  i only user my README.md for any non scriptable stuff (such as external services i rely on such as cloudflare/vpn providers, etc)
  
  source
Wxfisch@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
I’m curious how everyone documents their core/critical configs to allow the non-technical in our homes work with it if needed. For instance if I’m on work travel and the Pi-hole goes down for whatever reason my wife wouldn’t be able to use pretty much anything online. I can remote in and fix it but that could be hours/a day or two later. Same then for the proxmox stack that everything runs on.

Along the same lines, how are folks documenting for EOL? It may not be a happy thought but we are all going to go someday, so what is your plan and how have you ensured loved ones can access/save important data?

source
- BastingChemina@slrpnk.net ⁨5⁩ ⁨weeks⁩ ago
  It’s not just for my home server but for EOL or other issues I used bitwarden emergency access options for passwords. Of anything happens to me my wife can request access to my vault and if I don’t deny it in a certain timeframe she will have full access to it.
  
  I did that after my brother in law got in an accident and fell into a coma. I’m very grateful he had all his password saved in chrome on his unlocked laptop because if not it would have made the period insanely more difficult for my sister.
  
  Simple things like paying the bills would I been insanely more difficult and stressful and you don’t need extra stress in this period.
  
  source
- MangoPenguin@lemmy.blahaj.zone ⁨5⁩ ⁨weeks⁩ ago
  My solution is other people in the house don’t rely on anything in my setup, other than the router which runs some basic telemetry and fraud/phishing domain blocking but that’s all.
  
  source
30p87@feddit.org ⁨5⁩ ⁨weeks⁩ ago
I just try to make everything as self documented as possible. Eg. no stray scripts, everything is either implemented in other software (eg. my main wireguard peer in systemd-networkd, without any non-obvious external dependencies) or tracked as a part of an arch pkg with a simple PKGBUILD. I usually either have very simple nets (eg. my VPN’s net only consists of four peers, in the order main - secondary - laptop - phone) or leave it to DHCP and SLAAC. I try to avoid using ports for local servers as much as possible and use unix sockets. Stuff like LVM/Luks pretty much documents itself.

source
doodledup@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
I refused to do any documentation for a long time because it made me feel stupid for not memorizing it. I learned it the hard way… No I do everything I can with Git and Readmes.md.

source
Shmandom@feddit.uk ⁨5⁩ ⁨weeks⁩ ago
Secrets go in Keepass.

For server configs, a LibreOffice Writer file per machine (except for RPs, I only have one for those), written as a didactic manual explaining how to install and configure everything (I work on bare metal still). I started that way since diving into self-hosting was also a way to learn Linux, Sys-Admin and web-hosting. I don’t do anything without updating the relevant chapters, or creating new ones. Not gonna lie: it’s tedious. But also a life-saver, and the rationals for my choices remain available years later, which is priceless in many ways.

Once upon a time I had neat network gear running, and I mostly YOLO-ed the doc for those, relying instead on the firmware/config backups. I had to put those devices away, but when I finally get to play with them again, I’m going to suffer re-learning and re-discovering everything.

Recently, I got to hack and old console, and just did a chronological log-file with actions taken and URLs to guides, instead of writing down everything myself. It got me thinking I might add a simple log-file to track my actions, on top of my usual guides.

Or not. Having a life is nice too 😅

source
irmadlad@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
I am the note taking king probably. I worked in the construction industry for 20 years. The rule was, ‘if you didn’t write it down, it didn’t happen.’ That has just carried over to every other aspect of my life including selfhosting. Whenever I sit down to my terminal to do anything, I open Notepad++ and a regular windows notepad session. The windows notepad session is a little script I came up with that opens windows notepad with 1000 empty lines. It’s one of the many quirks I have, but I hate having to hit the enter key to start a new line. I like to be able to click on a new line for a new line of thought and start typing.

@echo off (for /l %%i in (1,1,1000) do echo.) > empty_lines.txt start notepad empty_lines.txt

(Save as a bat link on desktop)

Anyways, the Notepad ++ session is for after things get worked out, I make an official entry into the Notepad++. The windows notepad session is just a scratch pad or ‘thinking paper’ from which I transfer to the Notepad ++ doc. Convoluted, no? LOL You asked, and I just pulled back the curtain for you a bit. Careful what you ask for, could stain your brain.

I try to document everything. I feel like, if I’m going to take the time to learn something, I might as well write it down. I take my Grok sessions and distill them down if I found the info relevant. I also do all of this because after my TBI which gave me a seizure condition as well as other mental/neuro issues, my memory is shit, even for someone of my age bracket. But I can stand up a server and secure it, just from my notes in a step by step manner conducive to my limited mental acuity. I’ve often wondered if anyone would be interested in my notes, like maybe some newcomer to selfhosting wouldn’t have to reinvent the wheel since I have a penchant for fucking things up.
source
Nibodhika@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
I had that same problem, then I saw some YouTube videos where the guy recommended using Ansible to do stuff and it’s been night and day, not only it’s reproducible so if I ever want to move a service to another machine all I have to do is move a couple of roles around and possibly copy stuff over to keep the data but also it acts as documentation, because if I ever forget something I can look at the code.

Also I decided to write the roles myself instead of relying on pre-existing ones, so there’s some logic to how my stuff gets deployed and it’s easy to extend for any new stuff I want to add.

source
CapitalNumbers@lemm.ee ⁨5⁩ ⁨weeks⁩ ago
Here’s my approach to documentation. It’s about habits as much as it’s about actually writing anything down:

Never setup anything important via naked terminal commands that you will forget you did

Always wrap important commands in some kind of “setup-xyz.sh” script and then run that script to see if your install worked.

If you need to make a change to your service, ensure you update your script and so it can be re-run without braking anything

Get into the habit of this and you are documenting as you go
source
philpo@feddit.org ⁨5⁩ ⁨weeks⁩ ago
I use different things:

Netplan for the actual hard “inventory” like documentation. What cable goes where, what powers what, what MAC is assigned to what, when did I buy this or that?(The later with an addon). In theory I also have snipe-it,but doing it all in netplan is more convenient.

Wiki.js for the concept and How-To-Side. Lots of draw.io diagrams (which can be done directly in articles), HOWTOs as a reminder for me, naming conventions, etc. Also some basic inventory information for disaster recovery. (Wiki.js is not hosted locally)

Vaultwarden for all secrets,passwords, recovery keys, ssh keys,etc.

And last but not least I do backups to Mdisc Blue Rays every few months. These include the documentation, the most important files (knx project for example), etc. and are stored at a different location (bank safe). There is also an detailled explanation in both my wifes and my own will how to access these so if something happens the kiddos or someone taking care of them can gain access.
source
- arcayne@lemmy.today ⁨5⁩ ⁨weeks⁩ ago
  Did you mean Netbox?
  
  source
  - philpo@feddit.org ⁨5⁩ ⁨weeks⁩ ago
    Damn,yeah …it was late…
    
    source
swizzlestick@lemmy.zip ⁨5⁩ ⁨weeks⁩ ago
🧠 + a few slapdash notes in a password manager. It’s very organic, very human.

Occasionally leads to situations like this.

source
borax7385@lemmy.world ⁨5⁩ ⁨weeks⁩ ago
The Ansible playbooks I use to deploy it are the documentation.

source
truxnell@aussie.zone ⁨5⁩ ⁨weeks⁩ ago
Trillium notes

source
- fossphi@lemm.ee ⁨5⁩ ⁨weeks⁩ ago
  But what about the documentation to host the note server?
  
  source
  - truxnell@aussie.zone ⁨5⁩ ⁨weeks⁩ ago
    I run nixos, so my entire homelab servers setup is stored in my github. I can rebuild a server and restore backups from that (and sops keys). I significantly prefer this to having to remember and note a huge bunch of things to setup a computer - this way a rebuild on new oc is only a few hours tops
    
    source
tofu@lemmy.nocturnal.garden ⁨5⁩ ⁨weeks⁩ ago
I started to put it all in my selfhosted bookstack and that works well for me. I also automated a good part of my setup with Ansible, so I can just check how the Playbook did things if I forgot.

source
fmstrat@lemmy.nowsci.com ⁨5⁩ ⁨weeks⁩ ago
Every time I set up anything, I do one of two things:

If it’s container based, it gets a commented docker compose file in my custom orchestration

If it’s on a host system, the changes are scripted and commented in a setup script, which are run on new machines. If the acrit is specific to one machine, it is configured as such

I find in-setup docs to be best for a home lab, plus if I have to replace hardware, it’s fast.

Fun fact, I do it for laptops and desktops, too.
source
- CapitalNumbers@lemm.ee ⁨5⁩ ⁨weeks⁩ ago
  this is basically what i ended up doing to - glad to see my approach verified somewhat ha ha!
  
  but yeah, in general whenever i make a change / add new service, i always try and add those steps to some sort of setup.sh / docker-compose
  
  source
  - fmstrat@lemmy.nowsci.com ⁨5⁩ ⁨weeks⁩ ago
    Yea comes in super handy when you always want dropbear SSH for remote unlock, or making sure both RAID disks boot, etc.
    
    I do it for all my software setup, too. A shell script for each, then a for loop that asks to run each. But I also made github.com/fmstrat/gam, so maybe I just like overkill bash.
    
    source

-> View More Comments