Submitted 1 year ago by lepinkainen@lemmy.world to selfhosted@lemmy.world
https://docs.docker.com/docker-hub/usage/
AFAIK every NAS just uses unauthenticated connections to pull containers, I’m not sure how many actually allow you to log in even (raising the limit to a whopping 40 per hour).
So hopefully systems like /r/unRAID handle the throttling gracefully when clicking “update all”.
Anyone have ideas on how to set up a local docker hub proxy to keep the most common containers on-site instead of hitting docker hub every time?
Forgejo gives you a registry built-in.
Also is it just me or does the docker hub logo look like it’s giving us the middle finger?
I’m quite new to docker for NAS stuff - how many pulls would the average person do? like, i don’t think i even have 10 containers 🤨
I’m running ~30 containers, but they don’t typically all get new updates at the same time.
Updates are grabbed nightly, and I think the most I’ve seen update at once is like 6 containers.
Could be a problem for setting up a new system, or experimenting with new toys.
The problem is that the main container can (and usually does) rely on other layers, and you need to pull updates for those too. Updating one app can take 5-10 individual pulls.
Well shit, I still rely on Docker Hub even for automated pulls so this is just great. I guess i’m going back to managing VMs with OpenTofu and package managers.
The issue isn’t Docker vs Podman vs k8s vs LXC vs others. They all use OCI images to create your container/pod/etc. This new limit impacts all containerization solutions, not just Docker.
Instead, the issue is Docker Hub vs Quay vs GHCR vs others. It’s about where the OCI images are stored and pulled from. If the project maintainer hosts the OCI images on Docker Hub, then you will be impacted by this regardless of how you use the OCI images.
Some options include:
Lxc doesn’t use oci images? I always end up using docker in lxc when dockeris the only option (which I have not figured how to makw work on my airgapped side
Is there a project that acts like a registry? It can proxy the request with TTL, and you can push images to it too?
Artifactory is mandatory in some industries because it will keep all the versions of the images forever so that you can build your projects reliably without an internet connction.
Almost all of them. Forgejo handles containers already for example
Pull through Cache / proxy is what you’re looking for.
How? I was looking for this (although not very thoroughly)
I think most self-hosted Git+CI/CD platforms have container registry as a feature, but I’m not aware of a service that is just a standalone registry.
It’s easy to oversee because of the generic name, but this is pretty much that: hub.docker.com/_/registry
Use a service that’s not Docker hub
If only they used a distributed protocol like ipfs, we wouldn’t be in this situation
Huh. I was just considering establishing a caching registry for other reasons. Ferb, I know what we’re going to do today!
Same here. I’ve been building a bootstrap script, and each time I test it, it tears down the whole cluster and starts from scratch, pulling all of the images again. Every time I hit the Docker pull limit after 10 - 12 hours of work, I treat that as my “that’s enough work for today” signal. I’m going to need to set up a caching system ASAP or the hours I work on this project are about to suddenly get a lot shorter.
Do you have a good resource for how one can go about this?
You can host your own with harbor, and set up replication per repo (pull upstream tags) If you need a commercial product/support you can use MSR v4.
Harbor can install on any K8s cluster using helm, with just a couple of dependencies (cert-manager, postgres op, redis-op) Replication stuff you can easily add.
I have some no-warranty terraform I could share if there is some interest.
I’ve only done my “is it even possible” research so far, but these look promising:
How long since getting an oracle CEO did this take?
Did they really? Oh my god please tell me your joking, that a company as modern as docker got a freaking oracle CEO. They pulled a Jack Barker. Did he bring his conjoined triangles of success?
A “jack barker” 🤣
Fortunately linuxserver’s main hosting is no longer dockerhub.
Would you be able to share more info? I remember reading their issues with docker, but I don’t recall reading about whether or what they switched to. What is it now?
They run their own registry at lscr.io. You can essentially prefix all your existing linuxserver image names with lscr.io/ to pull them from there instead.
distribution.github.io/distribution/
is an opensource implementation of a registry.
you could also self host something like gitlab, which bundles this or sonatype nexus which can serve as a repository for several kinds of artifacts including container images.
Gitea and therefore Forgejo also have container registry functionality, I use that for private builds.
Jumping on the forgejo love train
Codeberg as woodpecker CI
oh, thats good to know, forgejo seems way nicer for self hosting than the limited gitlab open source core.
Instead of using a sort of Docker Hub proxy, you can also use GitHub’s repository or Quay. If the project allows it, you can easily switch to these alternatives. Alternatively, you can build the Docker image yourself from the source. It’s usually not a difficult process, as most of it is automated. Or what I personally would probably do is just update the image a day later if I hit the limit.
You can also host your own with harbor (or MSR v4 if you want a commercial product.) You can set them up to replicate upstream.
hedgehog@ttrpg.network 1 year ago
Do you mean a Docker container registry? If so, here are a couple options:
friend_of_satan@lemmy.world 1 year ago
I think they mean pull-through cache. shipyard.build/…/how-to-docker-registry-pull-thro…