Running DNS server in Docker

Submitted ⁨⁨4⁩ ⁨weeks⁩ ago⁩ by ⁨fendrax@jlai.lu⁩ to ⁨selfhosted@lemmy.world⁩

Hi everyone,

I own a homeserver running multiple services set up in individual docker compose projects. One of them is dnsmasq + pi-hole. Pi-hole is exposed on the host’s 53 port and uses dnsmasq as upstream. The host’s /etc/resolv.conf points to 127.0.0.1 to use pi-hole. Other containers from other compose projects on the same machine indirectly use pi-hole as well, as they use docker’s embedded DNS which forwards to the host.

Everything had worked fine for weeks before I ran into an issue after unattended-upgrades updated docker. After the upgrade, systemd attempted to restart the docker service. From the logs, it seems that the containers had a hard time honoring the SIGTERM. They started to complain about failing to reach the DNS server. At least that’s my understanding from journalctl -u docker. Here’s a (long) excerpt that shows:

containers failing to stop gracefully
stale processes after docker forcibly shut them down
repeated errors “stream copy error: reading from a closed fifo” afterwards that I am wondering what they mean

avril 12 06:46:24 prosper systemd[1]: Stopping docker.service - Docker Application Container Engine...
avril 12 06:46:24 prosper dockerd[1580]: time="2024-04-12T06:46:24.161850246+02:00" level=info msg="Processing signal 'terminated'"
avril 12 06:46:33 prosper dockerd[1580]: time="2024-04-12T06:46:33.276053218+02:00" level=error msg="[resolver] failed to query external DNS server" client-addr="udp:127.0.0.1:37197" dns-server="udp:127.0.0.1:53" error="read udp 127.0.0.>
avril 12 06:46:34 prosper dockerd[1580]: time="2024-04-12T06:46:34.509762706+02:00" level=info msg="ignoring event" container=d518bb77113596e96db39a6770fd9955ec29efacd907cdb02148708e92156cd5 module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:34 prosper dockerd[1580]: time="2024-04-12T06:46:34.989217237+02:00" level=info msg="ignoring event" container=81b3413f7d2409fe1a8fd8e1cf7dce3a23a6f7a4dc7cc23c13e9f1ba84b24eb7 module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:34 prosper dockerd[1580]: time="2024-04-12T06:46:34.989312976+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=642a3c9a2c821727f30c5cf84aac52778c027083f61d571831a408a2a8c>
avril 12 06:46:34 prosper dockerd[1580]: time="2024-04-12T06:46:34.989126980+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=781a3eac3f08d78ef285bc0231c00bd52005fe2257fd358f7c2b51af392>
avril 12 06:46:34 prosper dockerd[1580]: time="2024-04-12T06:46:34.989532834+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=7406c47622ecf7fb3f9e9ff6b7a8be2a3920342b4921b33a32b3df45123>
avril 12 06:46:34 prosper dockerd[1580]: time="2024-04-12T06:46:34.989693081+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=e5311ad000d63615c2336db1d3630cde233495108029648289f3660ad2d>
avril 12 06:46:34 prosper dockerd[1580]: time="2024-04-12T06:46:34.989723995+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=e6194d95ed1b95dddeaacab68f64a8e869c7a2489144198a84865d57f89>
avril 12 06:46:35 prosper dockerd[1580]: time="2024-04-12T06:46:35.350637865+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=249922bad7603a6e00bbcb0528cb7b7774a321f23e9f81b9784cc5968b5>
avril 12 06:46:35 prosper dockerd[1580]: time="2024-04-12T06:46:35.506956200+02:00" level=info msg="ignoring event" container=a5ac3703480cbfd4455d4d0b373023f730da63a8792863e19bfda469ee44370b module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:35 prosper dockerd[1580]: time="2024-04-12T06:46:35.602466692+02:00" level=info msg="ignoring event" container=1bd974e7ccf8f77ed93b51aa9eb0bed07cd0e26db7201fecb33c5c6f82712cb2 module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:35 prosper dockerd[1580]: time="2024-04-12T06:46:35.661037905+02:00" level=info msg="ignoring event" container=881fbf5034c7c0016b0715a8028e2c69788ddff218bc70fef98b30019d2b48bd module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:35 prosper dockerd[1580]: time="2024-04-12T06:46:35.714184730+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=0d9862444b4a7fce35df527f034e4f673aecf44a6551858056f9d8c7ce4>
avril 12 06:46:35 prosper dockerd[1580]: time="2024-04-12T06:46:35.775230427+02:00" level=error msg="[resolver] failed to query external DNS server" client-addr="udp:127.0.0.1:42577" dns-server="udp:127.0.0.1:53" error="read udp 127.0.0.>
avril 12 06:46:35 prosper dockerd[1580]: time="2024-04-12T06:46:35.881449888+02:00" level=info msg="Container failed to exit within 10s of signal 2 - using the force" container=c0147e337cee407e7ab9b133bd20667d287d6e2e121259c4794e4e090b12>
avril 12 06:46:35 prosper dockerd[1580]: time="2024-04-12T06:46:35.893446715+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=0eecbecfcaa2639946f0ed4a61a36f6ef004453af793cab91a52a97724d>
avril 12 06:46:35 prosper dockerd[1580]: time="2024-04-12T06:46:35.987621257+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=3493720ae2fc565fdc052a1d137590818d418d290f4418ec6911cac4704>
avril 12 06:46:36 prosper dockerd[1580]: time="2024-04-12T06:46:36.186176914+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=5cee8cfa0e068dbda35e89bcaec19450b50a0903b13bd677f6a6cb61686>
avril 12 06:46:36 prosper dockerd[1580]: time="2024-04-12T06:46:36.186467614+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:36 prosper dockerd[1580]: time="2024-04-12T06:46:36.186476595+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:36 prosper dockerd[1580]: time="2024-04-12T06:46:36.214354986+02:00" level=info msg="ignoring event" container=e6194d95ed1b95dddeaacab68f64a8e869c7a2489144198a84865d57f8970a3f module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:36 prosper dockerd[1580]: time="2024-04-12T06:46:36.339111902+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=8f42101141cf634f3495ae850b368a4b00ce065fb9a401ab7e1450d532b>
avril 12 06:46:36 prosper dockerd[1580]: time="2024-04-12T06:46:36.484546249+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=881fbf5034c7c0016b0715a8028e2c69788ddff218bc70fef98b30019d2>
avril 12 06:46:36 prosper dockerd[1580]: time="2024-04-12T06:46:36.728059697+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=1bd974e7ccf8f77ed93b51aa9eb0bed07cd0e26db7201fecb33c5c6f827>
avril 12 06:46:36 prosper dockerd[1580]: time="2024-04-12T06:46:36.753403840+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=d518bb77113596e96db39a6770fd9955ec29efacd907cdb02148708e921>
avril 12 06:46:36 prosper dockerd[1580]: time="2024-04-12T06:46:36.787737744+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=c39c900cb5ccd9ae3d48be4f953389edcab7bf051b9d505e23a816a8627>
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.235834609+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=13a3101e56e66f045318ad572a19b259a8cd3fbe7772b82f9262c15b727>
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.331772840+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=a5ac3703480cbfd4455d4d0b373023f730da63a8792863e19bfda469ee4>
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.411877163+02:00" level=warning msg="Health check for container e6194d95ed1b95dddeaacab68f64a8e869c7a2489144198a84865d57f8970a3f error: context canceled"
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.705699519+02:00" level=error msg="[resolver] failed to query external DNS server" client-addr="udp:127.0.0.1:58293" dns-server="udp:127.0.0.1:53" error="read udp 127.0.0.>
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.748247132+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=81b3413f7d2409fe1a8fd8e1cf7dce3a23a6f7a4dc7cc23c13e9f1ba84b>
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.790571926+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=e6e82f5a61e3b412c93abdd7fcc1cf25560e7d8ae957313fd36429f3ed7>
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.811976559+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=48d78ba0c55912bd57f72c6934940ad173261364d13483e630769b670d0>
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.832092046+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=0c7717003531fe815060e60da13065abe8399782c750d526e7fba872f0e>
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.965233556+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=73b5051331f81477c6358d74e889fedf50bf0c57696fdd8a735e5a20f3b>
avril 12 06:46:37 prosper dockerd[1580]: time="2024-04-12T06:46:37.965447255+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=c73fda2593bbbeb66589373add0609f8a165eab6d3ef4f0fe9bc871b1ca>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.094152959+02:00" level=info msg="ignoring event" container=0eecbecfcaa2639946f0ed4a61a36f6ef004453af793cab91a52a97724d5225f module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.113626463+02:00" level=info msg="ignoring event" container=642a3c9a2c821727f30c5cf84aac52778c027083f61d571831a408a2a8c2c71f module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.113684911+02:00" level=info msg="ignoring event" container=e5311ad000d63615c2336db1d3630cde233495108029648289f3660ad2d6c33a module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.192923515+02:00" level=info msg="ignoring event" container=8f42101141cf634f3495ae850b368a4b00ce065fb9a401ab7e1450d532b917ba module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.212691218+02:00" level=info msg="ignoring event" container=7406c47622ecf7fb3f9e9ff6b7a8be2a3920342b4921b33a32b3df45123878f2 module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.282823538+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=a8be522903b221bd8af2b6bf0533b56bc828863852d17fa604ed932d8cd>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.364351179+02:00" level=info msg="ignoring event" container=249922bad7603a6e00bbcb0528cb7b7774a321f23e9f81b9784cc5968b54ca54 module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.433220060+02:00" level=info msg="ignoring event" container=0d9862444b4a7fce35df527f034e4f673aecf44a6551858056f9d8c7ce44dbd4 module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.435651309+02:00" level=error msg="[resolver] failed to query external DNS server" client-addr="udp:127.0.0.1:59049" dns-server="udp:127.0.0.1:53" error="read udp 127.0.0.>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.503015026+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=6c0fcacc03cef94239fadb71e55ded7b01d07ed075023966427072d480a>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.531879355+02:00" level=info msg="Container failed to exit within 10s of signal 28 - using the force" container=6290112ab909221f1d3eee48a09e49403c1f61b7e1675ab3e581649ea82>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.789853647+02:00" level=info msg="Container failed to exit within 10s of signal 28 - using the force" container=89d8e915dc9ab961502830d0f8086d1a4b709d726d69e6d26570463219a>
avril 12 06:46:38 prosper dockerd[1580]: time="2024-04-12T06:46:38.928854916+02:00" level=info msg="ignoring event" container=5cee8cfa0e068dbda35e89bcaec19450b50a0903b13bd677f6a6cb6168675d21 module=libcontainerd namespace=moby topic=/tas>
avril 12 06:46:39 prosper dockerd[1580]: time="2024-04-12T06:46:39.089539561+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=b52727b63c13631c15b7b22d400f2c913ae73a2f5b36b0f566bef0ba645>
avril 12 06:46:39 prosper dockerd[1580]: time="2024-04-12T06:46:39.202791198+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=deda565519df8cd8705fff76564af4a084514fea41fe03ca1ab7e56f14b>
avril 12 06:46:39 prosper dockerd[1580]: time="2024-04-12T06:46:39.290584165+02:00" level=error msg="Force shutdown daemon"
avril 12 06:46:39 prosper dockerd[1580]: time="2024-04-12T06:46:39.567930258+02:00" level=info msg="Container failed to exit within 10s of signal 15 - using the force" container=c813f03c24c960653e28a9857a5d7f1fde677bcd3e226ed1585d41626a9>
avril 12 06:46:39 prosper dockerd[1580]: time="2024-04-12T06:46:39.725804932+02:00" level=info msg="Daemon shutdown complete"
avril 12 06:46:42 prosper systemd[1]: docker.service: Deactivated successfully.
avril 12 06:46:42 prosper systemd[1]: docker.service: Unit process 15700 (docker-proxy) remains running after unit stopped.
avril 12 06:46:42 prosper systemd[1]: docker.service: Unit process 15979 (docker-proxy) remains running after unit stopped.
avril 12 06:46:42 prosper systemd[1]: docker.service: Unit process 18294 (docker-proxy) remains running after unit stopped.
avril 12 06:46:42 prosper systemd[1]: Stopped docker.service - Docker Application Container Engine.
avril 12 06:46:42 prosper systemd[1]: docker.service: Consumed 3h 16min 43.639s CPU time.
avril 12 06:46:42 prosper systemd[1]: docker.service: Found left-over process 15700 (docker-proxy) in control group while starting unit. Ignoring.
avril 12 06:46:42 prosper systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
avril 12 06:46:42 prosper systemd[1]: docker.service: Found left-over process 15979 (docker-proxy) in control group while starting unit. Ignoring.
avril 12 06:46:42 prosper systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
avril 12 06:46:42 prosper systemd[1]: docker.service: Found left-over process 18294 (docker-proxy) in control group while starting unit. Ignoring.
avril 12 06:46:42 prosper systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
avril 12 06:46:42 prosper systemd[1]: Starting docker.service - Docker Application Container Engine...
avril 12 06:46:44 prosper dockerd[276530]: time="2024-04-12T06:46:44.366430751+02:00" level=info msg="Starting up"
avril 12 06:46:45 prosper dockerd[276530]: time="2024-04-12T06:46:45.443488954+02:00" level=info msg="[graphdriver] using prior storage driver: overlay2"
avril 12 06:46:57 prosper dockerd[276530]: time="2024-04-12T06:46:57.117503878+02:00" level=info msg="Loading containers: start."
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.071827546+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.074049075+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.077609642+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.077684622+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.082780007+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.084236865+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.088120145+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.088249509+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.168036186+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.168083531+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.169898516+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.171988424+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.172240688+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.172348881+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.173855919+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.176585486+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.178804059+02:00" level=info msg="ignoring event" container=781a3eac3f08d78ef285bc0231c00bd52005fe2257fd358f7c2b51af39266c42 module=libcontainerd namespace=moby topic=/t>
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.180166037+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.185876045+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.270156961+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.270186688+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.281083861+02:00" level=info msg="ignoring event" container=89d8e915dc9ab961502830d0f8086d1a4b709d726d69e6d26570463219a620f1 module=libcontainerd namespace=moby topic=/t>
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.282779586+02:00" level=info msg="ignoring event" container=6290112ab909221f1d3eee48a09e49403c1f61b7e1675ab3e581649ea829ad0f module=libcontainerd namespace=moby topic=/t>
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.298497572+02:00" level=info msg="ignoring event" container=b52727b63c13631c15b7b22d400f2c913ae73a2f5b36b0f566bef0ba645e7c0f module=libcontainerd namespace=moby topic=/t>
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.316205371+02:00" level=info msg="ignoring event" container=c73fda2593bbbeb66589373add0609f8a165eab6d3ef4f0fe9bc871b1ca315a7 module=libcontainerd namespace=moby topic=/t>
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.316278998+02:00" level=info msg="ignoring event" container=3493720ae2fc565fdc052a1d137590818d418d290f4418ec6911cac4704eb7e8 module=libcontainerd namespace=moby topic=/t>
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.341300109+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.341872295+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.370180971+02:00" level=info msg="ignoring event" container=6c0fcacc03cef94239fadb71e55ded7b01d07ed075023966427072d480ab42f7 module=libcontainerd namespace=moby topic=/t>
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.382180381+02:00" level=info msg="ignoring event" container=c0147e337cee407e7ab9b133bd20667d287d6e2e121259c4794e4e090b123e11 module=libcontainerd namespace=moby topic=/t>
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.436201339+02:00" level=error msg="stream copy error: reading from a closed fifo"
avril 12 06:46:58 prosper dockerd[276530]: time="2024-04-12T06:46:58.436352997+02:00" level=error msg="stream copy error: reading from a closed fifo"

I ssh’ed into the server and noticed that the pi-hole & dnsmasq containers were not running. I ran docker compose up -d in all projects, starting with the DNS one and it went back to normal.

My question is: how bad of an idea is this to run a DNS in docker and use it for the host and other containers? Maybe I just had back luck but I can’t help thinking this setup creates some sort of circular dependency between the host, docker and the containers.

If anyone has experience with a similar setup, I’d be curious to hear their opinion on the issue I ran into.

Thanks!

source

Comments

Sort:hotnew top

vegetaaaaaaa@lemmy.world ⁨4⁩ ⁨weeks⁩ ago
Not an answer but still relevant: I actively avoid enabling unattended-upgrades for third-party repositories like Docker (or anything that is not an official Debian repository) because they don’t have the same stability guarantees, and rely on other upgrade notification methods instead.

how bad of an idea is this to run a DNS in docker and use it for the host and other containers?

Personally I would simply install dnsmasq directly on the host because it is one apt install and a configuration file away. Keep it simple.

source
- fendrax@jlai.lu ⁨4⁩ ⁨weeks⁩ ago
  Yeah, that was my plan B. To be honest, I was not super confident that it would work when I put this setup together, because of the “host uses a container as DNS and docker uses the host as DNS” kind of circular dependency.
  
  But people do use docker for DNS servers so it has to work, right? That’s where I’d like to understand where I’m wrong. I’m fine with running pi hole and dnsmasq on the host as long as I get why this is not doable in docker.
  
  Thanks for your input, though. That’s helpful.
  
  source
  - vegetaaaaaaa@lemmy.world ⁨4⁩ ⁨weeks⁩ ago
    Usually you would have a second DNS resolver configured in /etc/resolv.conf (or whatever name resolution config system you are using, resolvconf, systemd-networkd, etc). The system will fall back to this resolver if the first resolver fails to respond (and/or replies NXDOMAIN, I’m not sure. The exact order and fallback conditions may vary depending on which system you use). This can be another dnsmasq instance, a public DNS resolver, your ISP’s resolver, etc. This allows at least basic DNS resolution to work before your dnsmasq instance comes back up.
    
    I would also add automatic monitoring for dnsmasq (either check that the service/container is running, or check the TCP connection to port 53, or check that DNS resolution is working for a known domain, etc)
    
    source
    -> View More Comments
lemmyvore@feddit.nl ⁨4⁩ ⁨weeks⁩ ago

circular dependency

What’s in/etc/resolv.conf inside the dns container?

What’s the upstream dns for the dns servers in the dns container?

source
- fendrax@jlai.lu ⁨4⁩ ⁨weeks⁩ ago
  In both the pi-hole (exposed on the host) and dnsmasq (used as upstream by pi-hole) containers:
  
  # Generated by Docker Engine. # This file can be edited; Docker Engine will not make further changes once it # has been modified. nameserver 127.0.0.11 options ndots:0 # Based on host file: '/etc/resolv.conf' (internal res olver) # ExtServers: [host(127.0.0.1)] # Overrides: [] # Option ndots from: internal
  
  So they are pointing to docker’s embedded DNS, itself forwarding to the host.
  
  source
  - lemmyvore@feddit.nl ⁨4⁩ ⁨weeks⁩ ago
    OK yeah, that might be a problem. You want to set the dns in the dnsmasq so it can resolve stuff upstream (probably with DoT or DoH), and the dns in pihole to the IP of dnsmasq. Look into the “dns”, “dns_search” and “dns_opt” compose directives.
    
    You may also want to consider simplifing your stack by using only dnsmasq or only pihole. Both of them can do pretty much the same things if you add some plugins.
    
    I’m using dnsmasq as lan dns server with DoH upstream as well as dhcp server. The dnsmasq is advertising itself as dns to all the dhcp clients. Also it runs on the router which allows it to hijack port 53 so any client attempting to do clear dns elsewhere ends up going through dnsmasq anyway. It also has an adblock plugin.
    
    source
    -> View More Comments
Smash@lemmy.self-hosted.site ⁨4⁩ ⁨weeks⁩ ago
I have two docker hosts which both run Pihole and get synced via gravity. I set those instances as first and second DNS servers in DHCP and on each docker host

source
TheHolm@aussie.zone ⁨4⁩ ⁨weeks⁩ ago
circular dependency seems to be the case. I guess adding second external resolver to /etc/resolve.conf will help. Second entry will not be used unless first one ( pi-hole) is responding. But it need to be tested.
BTW, why do you want to send host’s DNS via pihole?

source
- fendrax@jlai.lu ⁨4⁩ ⁨weeks⁩ ago
  Well, I have not really thought about why. I guess that’s partly due to old habits of running services on the host with systemd (my migration to docker is recent and still a work in progress). But I guess I’d like to continue to be able to resolve names of local devices on my network when connected through ssh on the host. Is that inherently wrong, still? I will implement the secondary DNS as a fallback. I am hoping to get rid of the issue that way.
  
  source
Willdrick@lemmy.world ⁨4⁩ ⁨weeks⁩ ago
Maybe a bit of a low tech solution, but I have an older RPI 3B running a second instance of PiHole.

source
Decronym@lemmy.decronym.xyz ⁨4⁩ ⁨weeks⁩ ago
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

Fewer Letters More Letters

DNS Domain Name Service/System

IP Internet Protocol

TCP Transmission Control Protocol, most often over IP

[Thread #683 for this sub, first seen 16th Apr 2024, 11:25] [FAQ] [Full list] [Contact] [Source code]

source
daddy32@lemmy.world ⁨4⁩ ⁨weeks⁩ ago
Shouldn’t you just have a fallback DNS for cases like these? During the outage, it would be the one used and after the things settle, you would be back at primary.

Maybe some kind of monitoring/notifications on top of thatm

source
- AtariDump@lemmy.world ⁨4⁩ ⁨weeks⁩ ago
  When it comes to a “secondary”DNS… [there is nothing like a primary and secondary DNS server. These indications are quite misleading but many systems adopted it this way. Pihole only list the DNS servers as primary and secondary, because this is what the providers write on their pages. The bad phrasing is supported especially by how Windows handles it.](https://discourse.pi-hole.net/t/primary-vs-secondary-dns/1536/2)
  
  [Most operating systems implement DNS servers as alternatives, not as fallbacks, i.e. they will query any of both servers from time to time, so it is quite likely that you will loose your Pi-hole filtering capabilities (at least partially) [if you specify a secondary DNS server on your network].](https://discourse.pi-hole.net/t/secondary-dns-server-for-dhcp/1874)
  
  The **ONLY** DNS server you should have set on your network is a/the PiHole(s).
  
  source
  - daddy32@lemmy.world ⁨4⁩ ⁨weeks⁩ ago
    Oh my, TIL. But what then in cases like these, when the Pihole itself is down? You need fallback…
    
    source
    -> View More Comments
- fendrax@jlai.lu ⁨4⁩ ⁨weeks⁩ ago
  Yes, others have already suggested something similar. I’ll do that first because it is easy. Monitoring-wise, I should already be covered but since prometheus is running on the same server, it was down during the outage. There is room for improvement, for sure! I have a couple of RPis on my network that I can leverage for better monitoring.
  
  source

Fewer Letters	More Letters
DNS	Domain Name Service/System
IP	Internet Protocol
TCP	Transmission Control Protocol, most often over IP