Every single time I think of restructuring my homelab storage. What do you use for storage engines and how does it benefit you?

Comments

• squinky@sh.itjust.works ⁨4⁩ ⁨hours⁩ ago

  Just btrfs.

  source

  • MrModest@lemmy.world ⁨1⁩ ⁨hour⁩ ago

    Why btrfs and not ZFS? In my info bubble, the btrfs has a reputation of an unstable FS and they ended up with unrecoverable data.

    source
• Cyber@feddit.uk ⁨2⁩ ⁨hours⁩ ago

  Backups… with LVM, if you’re trying to do a full system backup (ie with clonezilla, etc) then you have to backup the whole thing - you can’t backup just 1 drive.

  I have a media server with 2x 2TB HDDs and 1x SSD in a LVM, split into Music, Video, TV… and the OS … and I can backup the individual files of course, but I can’t backup just the OS drive.

  btrfs didn’t exist when I created it, but I use it on my NAS and it’s great.

  I’ll be rebuilding my media server one day and change LVM to btrfs.

  source
• panda_abyss@lemmy.ca ⁨18⁩ ⁨hours⁩ ago

  I set up garage, which works fine.

  Advantage of an s3 style layer is it’s simplicity and integration with apps.

  I also use it so I can run AI agents that have zero access to a task fine system

  source

  • possiblylinux127@lemmy.zip ⁨2⁩ ⁨hours⁩ ago

    Just out of curiosity, what are you using S3 for?

    source
• Dalraz@lemmy.ca ⁨18⁩ ⁨hours⁩ ago

  This has been my journey.

  I started with pure docker and hostpath on an Ubuntu server. This worked well for me for many years and is good for most people.

  Later I really wanted to learn k8s so I built a 3 node cluster with NSF managed PVC for storage, this was fantastic for learning. I enjoyed this for 3 plus years. This is all on top of proxmox and zfs

  About 8 months ago I decided I’m done with my k8s learning and I wanted more simplicity in my life. I created a lxc docker and slowly migrated all my workloads back to docker and hostpath, this time backed by my mirrored zfs files system.

  I guess my point is what are you hoping to get out of your journey and then tailor your solution to that.

  Also I do recommend using proxmox and zfs.

  source
• pokexpert30@jlai.lu ⁨12⁩ ⁨hours⁩ ago

  Longhorn is goated to manage volume availability across geographically distant nodes.

  If you’re running a one-node show, hostpath will do fine (or just dont kubernetes at all, tbh)

  source
• entropicdrift@lemmy.sdf.org ⁨17⁩ ⁨hours⁩ ago

  I just use mergerfs and SnapRAID so I can scale dynamically when I can afford new drives. Granted it’s all fully replaceable media files on my end, so I’m not obsessed with data integrity.

  source

  • rcmd@lemmy.world ⁨14⁩ ⁨hours⁩ ago

    Well, this path seems to be the most appropriate for what I am for.

    And more to that, both mergerfs and snapraid are available out of the box in the latest stable Debian release.

    Thanks for pointing me at it!

    source

    • signalsayge@infosec.pub ⁨7⁩ ⁨hours⁩ ago

      This is what I’m doing as well. The nice thing about it is that it supports different sized drives in the same mergerfs mount and with snapraid, you just need to make sure one of your biggest drives is the parity drive. I’ve got 10 drives right now with 78TB usable in the mergerfs mount and two 14TB drives acting as parity. I’ve been able to build it up over the years and add slowly.

      source
    • entropicdrift@lemmy.sdf.org ⁨13⁩ ⁨hours⁩ ago

      Happy to help!

      source
• skilltheamps@feddit.org ⁨16⁩ ⁨hours⁩ ago

  You need to ask yourself what properties you want in your storage, then you can judge which solution fits. For me it is:

  • effortless rollback (i.e. in case something with a db updates, does a db migration and fails)
  • effortless backups, that preserve database integrity without slow/cumbersome/downtime-inducing crutches like sql dump
  • a scheme that works the same way for every service I host, no tailored solutions for individual services/containers
  • low maintenance

  The amount of data I’m handling fits on larger harddrives (so I don’t need pools), but I don’t want to waste storage space. And my homeserver is not my learn and break stuff environment anymore, but rather just needs to work.

  I went with btrfs raid 1, every service is in its own subvolume. The containers are precisely referenced by their digest-hashes, which gets snapshotted together with all persistent data. So every snapshot holds exactly the amount of data that is required to do a seamless rollback. Snapper maintains a timeline of snapshots for every service. Updating is semi-automated where it does snapshot -> update digest hash from container tags -> pull new images -> restart service. Nightly offsite backups happen with btrbk, which mirrors snapshots in an incremental fashion on another offsite server with btrfs.

  source
• i_am_not_a_robot@discuss.tchncs.de ⁨17⁩ ⁨hours⁩ ago

  You can use OpenEBS to provision and manage LVM volumes. Host path requires you to manually manage the host paths.

  source