How to manage docker compose apps?

Comments

• hoppolito@mander.xyz ⁨3⁩ ⁨weeks⁩ ago

  But, how do folks manage this mess?

  I generally find it less of a mess to have everything encapsulated in docker deployments for my server setups. Each application has its own environment (i.e. I can treat each container as its own ‘Linux machine’ which has only the stuff installed that’s important) and they can all be interfaced with through the same cli.

  Is there an analogue to apt update, apt upgrade, systemctl restart, journalctl?

  Strictly speaking docker pull <image>, docker compose up, docker restart <container>, and docker logs <container>. But instead of finding direct equivalents to a package manager or system service supervisor, i would suggest reading up on

  1. the docker command line, with its simple docker run command and the (in the beginning) super important docker ps
  2. The concept of Dockerfiles and what exactly they encapsulate - this will really help understand how docker abstracts from single app messiness
  3. docker-compose to find the equivalent of service supervision in the container space

  Applications like immich are multi-item setups which can be made much easier while maintaining flexibility with docker-compose. In this scenario you switch from worrying about updating individual packages, and instead manage ‘compose files’, i.e. clumps of programs that work together to provide a specific service.

  Once you grok the way compose files make that management easier - since they provide the same isolation and management regardless of any outer environment, you have a plethora of tools that make manual maintenance easy (dockge, portainer,…) or, more often, make manual maintenance less necessary through automation (watchtower, ansible, komodo,…).

  I realise this can be daunting in the beginning but it is the exact use case for never having to think about downloading a new Go binary and setting up a manual unit file again.

  source
• hperrin@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago

  Don’t auto update. Read the release notes before you update things. Sometimes you have to do some things manually to keep from breaking things.

  source

  • suicidaleggroll@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

    Pretty much guaranteed you’ll spend an order of magnitude more time (or more) doing than than just auto-updating and fixing things on the rare occasion that they break.

    source

    • ChapulinColorado@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

      For people living with others it might not be a choice though. The lights not working for a day the way they normally do is all it takes for someone to lose all faith in automation. It’s easier when you plan for a specific time and day to update things.

      source
  • zingo@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

    Politically correct of course.

    But from my own experience using Watchtower for over 7 years is that I can count on one hand when it actually broke something. Most of the time it was database related.

    But you can put apps on the watchtower ignore list (looking a you Immich!), which clear that out fairly quick.

    source

    • WhatAmLemmy@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

      And if you roll all your dockers on ZFS as datasets + sanoid you can just rollback to the last snapshot, if that ever does happen.

      source
  • possiblylinux127@lemmy.zip ⁨3⁩ ⁨weeks⁩ ago

    Autoupdate is fine for personal stuff. Just set a specific date so that you know if something breaks. Rollbacks are easy and very rarely needed.

    source
• frongt@lemmy.zip ⁨3⁩ ⁨weeks⁩ ago

  I just use watchtower to update automatically.

  Docker has a logs command.

  source

  • antifa_ceo@lemmy.ml ⁨3⁩ ⁨weeks⁩ ago

    And being able to opt in just with a container label is super convenient

    source
• suicidaleggroll@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  Docker is far cleaner than native installs once you get used to it. Yes native installs are nice at first, but they aren’t portable, and unless the software is built specifically for the distro you’re running you will very quickly run into dependency hell trying to set up your system to support multiple services that all want different versions of libraries. Plus what if you want or need to move a service to another system, or restore a single service from a backup? Reinstalling a service from scratch and migrating over the libraries and config files in all of their separate locations can be a PITA.

  It’s pretty much a requirement to start spinning up separate VMs for each service to get them to not interfere with each other and to allow backup and migration to other hosts, and managing 50 different VMs is much more involved and resource-intensive than managing 50 different containers on one machine.

  docker compose pull/up/down is pretty much all you need, wrap it in a small shell script and you can bring up/down or update every service with a single command. Also if you use bind mounts and place them in the directory for the service along side the compose file, now your entire service is self-contained in one directory. To back it up you just “docker compose down”, rsync the directory to the backup location, then “docker compose up”. To restore you do the exact same thing, just reverse the direction of the rsync. To move a service to a different host, you do the exact same thing, just the rsync and docker compose up are now being run on another system.

  Docker lets you compact an entire service with all of its dependencies, databases, config files, and data, into a single directory that can be backed up and/or moved to any other system with nothing more than a “down”, “copy”, and “up”, with zero interference with other services running on your system.

  source

  • source_of_truth@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

    Can you explain the bind mounts part a bit please?

    source

    • suicidaleggroll@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

      There are two ways to maintain a persistent data store for Docker containers: bind mounts and docker-managed volumes.

      A Docker managed volume looks like: datavolume:/data And then later on in the compose file you’ll have volumes: datavolume:

      When you start this container, Docker will create this volume for you in /var/lib/docker/volumes/ and will manage access and permissions. They’re a little easier in that Docker handles permissions for you, but they’re also kind of a PITA because now your compose file and your data are split apart in different locations and you have to spend time tracking down where the hell Docker decided to put the volumes for your service, especially when it comes to backups/migration.

      A bind mount looks like: ./datavolume:/data

      When you start this container, if it doesn’t already exist, “datavolume” will be created in the same location as your compose file, and the data will be stored there. This is a little more manual since some containers don’t set up permissions properly and, once the volume is created, you may have to shut down the container and then chown the volume so it can use it, but once up and running it makes things much more convenient, since now all of the data needed by that service is in a directory right next to the compose file.

      source
• JASN_DE@feddit.org ⁨3⁩ ⁨weeks⁩ ago

  Check out Dockge

  source

  • ook@discuss.tchncs.de ⁨3⁩ ⁨weeks⁩ ago

    Was looking if anyone mentioned it!

    I started with portainer but it was way too complex for my small setup. Dockge works super well, starting, stopping, updating containers in a simple web interface.

    Just updating Dockge itself from inside Dockge does not seem to work but to be fair I didn’t look into it that much yet.

    source

    • mbirth@lemmy.ml ⁨3⁩ ⁨weeks⁩ ago

      Can Dockge manage/cleanup unused images and containers by now? That’s the only reason I keep using Portainer - because it can show all the other stuff and lets me free up space.

      source

      • -> View More Comments
• GreenKnight23@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  docker compose CLI.

  KISS, never did me wrong.

  source
• gravitywell@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

  I use Dockge to manage everything.

  source

  • Kyle@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago

    Wow thank you for this. This looks so much nicer than portainer.

    Subscribing to these communities is so helpful because of discovery like this.

    source
  • princessnorah@lemmy.blahaj.zone ⁨3⁩ ⁨weeks⁩ ago

    +1 for Dockge.

    source
  • lka1988@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

    Same here. It’s so much easier to use than Portainer, no weird licensing shit, and works parallel to existing stacks. Also helps me keep Docker stacks organized.

    I have 4 VMs on my cluster specifically for Docker, each with its own Dockge instance, which can all be linked together to access all Docker stacks from any instance.

    source
• SnotFlickerman@lemmy.blahaj.zone ⁨3⁩ ⁨weeks⁩ ago

  In the docker folder with the docker-compose.yml:

  docker compose pull && docker compose up -d

  As others have said, for large groups of containers it’s helpful to use Watchtower.

  Immich in particular warns to backup your database before an upgrade.

  Oh and after upgrades to remove any dangling images which sometimes take up a lot of space:

  docker image prune

  source

  • sobchak@programming.dev ⁨3⁩ ⁨weeks⁩ ago

    I think compose files are usually pinned to a version, or use a .env file that needs to be changed to update to a new version.

    I personally don’t update very often; usually not until I’m forced to for some reason. I find that just checking the documentation for any upgrade/migration guides, and doing it manually is sufficient. I don’t expose this kind of stuff publicly; if I did, I’d probably update regularly.

    source

    • SnotFlickerman@lemmy.blahaj.zone ⁨3⁩ ⁨weeks⁩ ago

      Immich is a more touchy beast because it includes a mobile app and the mobile app and the docker container need to generally be either the same version, or within a few versions of one another. There was a while where I forgot to update the server for a while and the mobile app kept being updated on my phone and stopped backing up photos because it could no longer communicate with the server.

      source
• communism@lemmy.ml ⁨3⁩ ⁨weeks⁩ ago

  Watchtower for automated updates. For containers that don’t have a latest tag to track, editing the version number manually and then docker compose pull && docker compose up -d is simple enough.

  source

  • kata1yst@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

    Adding here. Most docker containers support semver pinning! It’s a great balance between automated updates and advoiding breakage.

    source
• slazer2au@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  Each app has a folder and then I have a bash script that runs

  Docker compose up -d 
  

  In each folder of my containers to update them. It is crude and will break something at some stage but meh jellyseer or TickDone being offline for a bit is fine while I debug.

  source
• eager_eagle@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  Yeah, I have everything as compose.yaml stacks and those stacks + their config files are in a git repo.

  source
• LycaKnight@infosec.pub ⁨3⁩ ⁨weeks⁩ ago

  I use dockge. 1-2 years ago i started to selfhost everything with Docker. I have now 30+ Container and Dockge is absolut fantastic. I host all my stuff on a root Server from Hetzner and if they reveal a cheaper Server i switch. Since all my Stuff is hosted on Docker i can simple copy it to the new Server and start the Docker Containers and it runs.

  github.com/louislam/dockge

  source
• UnityDevice@lemmy.zip ⁨2⁩ ⁨weeks⁩ ago

  I use quadlets instead - it’s part of podman and lets you manage containers as systemd services. Supports automatic image updates and gives you all the benefits of systemd service management. There’s a tool out there that will convert a docker compose config into quadlet unit files giving you a quick start, but I just write them by hand.

  source
• Passerby6497@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  Docker compose pull; docker compose down;docker compose up -d

  Pulls an update for the container, stops the container and then restarts it in the background. I’ve been told that you don’t need to bring it down, but I do it so that even if there isn’t an update, it still restarts the container.

  You need to do it in each container’s folder, but it’s pretty easy to set an alias and just walk your running containers, or just script that process for each directory. If you’re smarter than I am, you could get the list from running containers (docker ps), but I didn’t name my service folders the same as the service name.

  source

  • paequ2@lemmy.today ⁨2⁩ ⁨weeks⁩ ago

    What commands do you have to run after you update docker-compose.yml or .env files? I updated one of those files once bad things happened… I haven’t had to update the configs in a long time.

    source

    • Passerby6497@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

      I usually just do

      Docker compose down
      Docker compose up -d
      

      As I would with any service restart. The up -d command is supposed to reload it as well, but I prefer knowing for certain that the service restarted.

      Out of curiosity, what did you update and what broke? I had that happen a lot when I was first getting started with docker, and is part of how I learned. Once you have a basic template (or have dec supplies example files), it makes spinning up new services less of a hassle.

      Though I still get yelled at about the version entry in my fines because I haven’t touched mine in forever

      source

      • -> View More Comments
• ikidd@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  I never, ever use a bare docker command. Other than actively working on a project, I can’t see why anyone would use that.

  Docker compose for every stack, watchtower for the containers I’m not too worried about breaking changes on update.

  source
• tofu@lemmy.nocturnal.garden ⁨3⁩ ⁨weeks⁩ ago

  I’d suggest to put the compose stacks in git and then clone them either manually or with some tool.

  For fully automated gitops with docker compose, check this blogpost

  source
• Chewy7324@discuss.tchncs.de ⁨3⁩ ⁨weeks⁩ ago

  It won’t save you from doing a bit of work but you could use podman. There’s systemd integration so you can still start/stop/enable your services with systemctl while using docker/container images. You won’t be able to use docker-compose directly, but it’s usually not that hard to replicate the logic with systemd (Immich was a PITA at first, but it improved considerably).

  I do this with NixOS quite a bit, and I’ve yet to use docker compose (although the syntax is different, it’s still the same process).

  source
• tehWrapper@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  I have finally had to switch to using docker for several things I use to just install manually (ttrss being the main one). It sure feels dirty when i use to just apt update and know everything was updated.

  I can see the draw for docker but feel it’s way over used right now.

  source

  • Jakeroxs@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

    Just replace Apt update with docker pull 🤷‍♂️

    source

    • tehWrapper@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

      Docker pull lacks a lot of the automation apt has if your using multiple images, and needing to restart them.

      I’m an old man set in my ways… i see the benefit of docker and having set images, but also see so much waste having multi installs of the same thing for different docker images.

      source

      • -> View More Comments
• HelloRoot@lemy.lol ⁨3⁩ ⁨weeks⁩ ago

  I manage them with dokploy.com

  source
• possiblylinux127@lemmy.zip ⁨3⁩ ⁨weeks⁩ ago

  Why wouldn’t you just use Docker compose? It has NFS support build in and there are Ansible playbooks for it

  source
• synae@lemmy.sdf.org ⁨3⁩ ⁨weeks⁩ ago

  I use k3s and argocd to keep everything synced to the configuration checked into a git repo. But I wouldn’t recommend that to someone just getting started with containers; kubernetes is a whole new beast to wrestle with.

  source

  • possiblylinux127@lemmy.zip ⁨3⁩ ⁨weeks⁩ ago

    Kubernetes is the Arch of Containers except way more confusing

    source

    • synae@lemmy.sdf.org ⁨3⁩ ⁨weeks⁩ ago

      I use it for work so it felt natural to do it at home too. If anyone has time to learn it as a hobby and doesn’t mind a challenge, I recommend it. But IMO you need to already be familiar with a lot of containerization concepts

      source
• Dagnet@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  Strongly recommend komodo. I tried dockge and portainer but komodo was easy to install and has great features from scheduled updates and using compose files from a git repo. Also you can migrate existing apps to it without too much work

  source

  • Bakkoda@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

    I just made the switch to Komodo last week. Komodo lxc managing 4 VMs across two proxmox hosts. Easily added all the existing servers and it just worked. I think any of these systems are probably overkill for my needs but Komodo had the nicest “fresh out the box, find the important stuff right away” feel to it. My two cents.

    source
• eksb@programming.dev ⁨3⁩ ⁨weeks⁩ ago

  I have 5 docker-compose-based services. I wrote a shell script:

  #!/usr/bin/env bash
  for y in $(find /etc/ -name docker-compose.yml); do
    cd $(dirname $y)
    docker compose pull
    systemctl resteart $y
  done
  

  source

  • Unquote0270@programming.dev ⁨3⁩ ⁨weeks⁩ ago

    I hope the real version doesn’t have the spelling problem!

    source
  • qqq@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

    For loops with find are evil for a lot of reasons, one of which is spaces:

    $ tree
    .
    ├── arent good with find loops
    │   ├── a
    │   └── innerdira
    │       └── docker-compose.yml
    └── dirs with spaces
        ├── b
        └── innerdirb
            └── docker-compose.yml
    
    3 directories, 2 files
    $ for y in $(find .); do echo $y; done
    .
    ./are
    t good with fi
    d loops
    ./are
    t good with fi
    d loops/i
    
    erdira
    ./are
    t good with fi
    d loops/i
    
    erdira/docker-compose.yml
    ./are
    t good with fi
    d loops/a
    ./dirs with spaces
    ./dirs with spaces/i
    
    erdirb
    ./dirs with spaces/i
    
    erdirb/docker-compose.yml
    ./dirs with spaces/b
    

    You can kinda fix that with IFS:

    $ OIFS=$IFS
    $ IFS=$'\n'
    $ for y in $(find .); do echo "$y"; done
    .
    ./arent good with find loops
    ./arent good with find loops/innerdira
    ./arent good with find loops/innerdira/docker-compose.yml
    ./arent good with find loops/a
    ./dirs with spaces
    ./dirs with spaces/innerdirb
    ./dirs with spaces/innerdirb/docker-compose.yml
    ./dirs with spaces/b
    $ IFS=$OIFS
    

    But you can also use something like:

    find . -name 'docker-compose.yml' -printf '%h\0' | xargs -0 ...
    

    or in your case this could work:

    find . -name 'docker-compose.yml' -execdir ...
    

    source
• non_burglar@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

  I didn’t see ansible as a solution here, which I use. I run docker compose only. Each environment is backed up nightly and monitored. If a docker compose pull/up and then image clean breaks a service, I restore from a backup that works and see what went wrong.

  source
• Jakeroxs@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

  It’s really nice once it’s going, especially if you link them together in a compose and farm out all the individual ymls for each service, or use something like dockage to do it.

  source
• TakenDistance@sh.itjust.works ⁨3⁩ ⁨weeks⁩ ago

  I use Portainer (portainer.io) - it’s a prett nice WebUI which lets me manage the whole set of services and easily add new ones as you can edit the compose yaml right in the browser.

  There’s no substitute for knowing all the docker commands to help you get around but if you are looking for something to help with management then this might be the way to go.

  Watchtower also recommended is probably a good shout just be warned about auto upstating past your config - it’s super easy for the next image you pull to just break your setup because of new ENV or other dependency you’re not aware of.

  source
• rozodru@piefed.social ⁨3⁩ ⁨weeks⁩ ago

  I run Akkoma, Navidrome, Searx, valutwarden, RomM, Forgejo, wireguard, RDP, and a few other things all via docker. Honestly I just keep everything in their own dir and just have Yazi on my server to make it easier to manage. I don’t auto update anything, it’s all manual updates.

  I’m probably going to slap Watchtower in there to just make things easier. don’t really need to over think it in all honesty.

  source