podman-compose icon indicating copy to clipboard operation
podman-compose copied to clipboard

DNS resolution fails for the container network.

Open allkhor opened this issue 10 months ago • 5 comments

Describe the bug DNS resolution fails for the container network.

Wordpress service from example examples/wordpress/docker-compose.yaml does not connect to the database.

Additionally, I noticed that in the bad commit, the file /run/user/1000/rootless-cni/run/user/1000/containers/cni/dnsname/dev_default/dnsmasq.conf does not appear.

After using git bisect, a regression was found from commit 978a138. With commit 4a232f5e, the issue is resolved.

To Reproduce Steps to reproduce the behavior:

  1. Download docker-compose.yaml
  2. Navigate to the folder containing the docker-compose.yaml file.
  3. Run podman-compose up.

Expected behavior The Wordpress service should connect to Mysql.

Actual behavior Containers do not resolve DNS. The Wordpress service cannot connect to mysql.

Environment:

  • OS: Linux Ubuntu 22.04.5 LTS, kernel version 6.8.0-52-generic
  • podman version 3.4.4
  • containernetworking-plugins 1.1.1+ds1-3build1
  • golang-github-containernetworking-plugin-dnsname 1.3.1+ds1-2ubuntu0.22.04.1

allkhor avatar Feb 27 '25 16:02 allkhor

With master version now (updated 2025 August), the wordpress example seems to work.

podman-compose version 1.5.0 podman version 5.4.1

whym avatar Sep 05 '25 12:09 whym

In my system, bug is still reproduced in master, with commit a1f3befa8ea.

allkhor avatar Sep 28 '25 09:09 allkhor

Is it reproducible with newer podman, like 5.4.1? Podman is developed and distributed separately from podman-compose.

whym avatar Sep 28 '25 09:09 whym

Sorry, this is my current working machine, and Podman 3.4.4 is the latest version in the package repositories. I will test it as soon as possible on an RPi 3 with Podman 4.3.1. Interestingly, on the main branch at commit  a1f3befa8ea, if I revert commit 978a1381bcab924a and resolve the conflicts, this fixes the bug and the WordPress example works as expected.

allkhor avatar Sep 28 '25 10:09 allkhor

Can't reproduce on:

  • Debian GNU/Linux 12 (Bookworm), kernel 6.12.47+rpt‑rpi‑v8
  • Podman Compose 1.5.0
  • Podman 4.3.1

allkhor avatar Sep 29 '25 07:09 allkhor

i'm getting this same error

the docker file:

wget -O docker-compose.yml https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml

i believe the dns lookup goes to aardvark but can't register the immich database record

trash@pchan:~/immich$ sudo podman exec -it immich_server sh -c "cat /etc/hosts" 
127.0.0.1 pchan localhost.localdomain
localhost ::1 localhost localhost.localdomain 
10.89.x.x host.containers.internal host.docker.internal 
10.89.x.x 02fcee9c8425 immich_server
trash@pchan:~/immich$ podman-compose --version
podman-compose version 1.5.0
podman version 5.6.2
[database]                | Using SSD storage
[redis]                   | 1:M 16 Dec 2025 02:33:56.369 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
[redis]                   | 1:M 16 Dec 2025 02:33:56.375 * oO0OoO0OoO0Oo Valkey is starting oO0OoO0OoO0Oo
[redis]                   | 1:M 16 Dec 2025 02:33:56.375 * Valkey version=8.1.4, bits=64, commit=00000000, modified=0, pid=1, just started
[redis]                   | 1:M 16 Dec 2025 02:33:56.375 # Warning: no config file specified, using the default config. In order to specify a config file use valkey-server /path/to/valkey.conf
[redis]                   | 1:M 16 Dec 2025 02:33:56.377 * monotonic clock: POSIX clock_gettime
[redis]                   | 1:M 16 Dec 2025 02:33:56.379 * Running mode=standalone, port=6379.
[redis]                   | 1:M 16 Dec 2025 02:33:56.380 * Server initialized
[redis]                   | 1:M 16 Dec 2025 02:33:56.380 * Ready to accept connections tcp
[database]                |
[database]                | PostgreSQL Database directory appears to contain a database; Skipping initialization
[database]                |
[database]                | 2025-12-16 02:33:56.525 GMT [1] LOG:  skipping missing configuration file "/var/lib/postgresql/data/postgresql.override.conf"
[database]                | 2025-12-16 02:33:56.526 GMT [1] LOG:  skipping missing configuration file "/var/lib/postgresql/data/postgresql.override.conf"
[database]                | 2025-12-16 02:33:56.719 UTC [1] LOG:  starting PostgreSQL 14.19 (Debian 14.19-1.pgdg12+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14+deb12u1) 12.2.0, 64-bit
[database]                | 2025-12-16 02:33:56.720 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
[database]                | 2025-12-16 02:33:56.720 UTC [1] LOG:  listening on IPv6 address "::", port 5432
[database]                | 2025-12-16 02:33:56.725 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
[database]                | 2025-12-16 02:33:56.737 UTC [27] LOG:  database system was shut down at 2025-12-16 02:33:01 UTC
[database]                | 2025-12-16 02:33:56.755 UTC [1] LOG:  database system is ready to accept connections
[immich-server]           | Initializing Immich v2.3.1
[immich-server]           | Detected CPU Cores: 1
[immich-machine-learning] | [12/16/25 02:34:00] INFO     Starting gunicorn 23.0.0
[immich-machine-learning] | [12/16/25 02:34:00] INFO     Listening at: http://[::]:3003 (10)
[immich-machine-learning] | [12/16/25 02:34:00] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[immich-machine-learning] | [12/16/25 02:34:00] INFO     Booting worker with pid: 18
[immich-server]           | Missing history for endpoint: Retrieve auth status
[immich-server]           | (node:2) ExperimentalWarning: WASI is an experimental feature and might change at any time
[immich-server]           | (Use `node --trace-warnings ...` to show where the warning was created)
[immich-machine-learning] | [12/16/25 02:34:07] INFO     generated new fontManager
[immich-machine-learning] | [12/16/25 02:34:07] INFO     Started server process [18]
[immich-machine-learning] | [12/16/25 02:34:07] INFO     Waiting for application startup.
[immich-machine-learning] | [12/16/25 02:34:07] INFO     Created in-memory cache with unloading after 300s
[immich-machine-learning] |                              of inactivity.
[immich-machine-learning] | [12/16/25 02:34:07] INFO     Initialized request thread pool with 8 threads.
[immich-machine-learning] | [12/16/25 02:34:07] INFO     Application startup complete.
[immich-server]           | node:internal/process/promises:394
[immich-server]           |     triggerUncaughtException(err, true /* fromPromise */);
[immich-server]           |     ^
[immich-server]           |
[immich-server]           | Error: getaddrinfo EAI_AGAIN database
[immich-server]           |     at GetAddrInfoReqWrap.onlookupall [as oncomplete] (node:dns:122:26) {
[immich-server]           |   errno: -3001,
[immich-server]           |   code: 'EAI_AGAIN',
[immich-server]           |   syscall: 'getaddrinfo',
[immich-server]           |   hostname: 'database'
[immich-server]           | }
[immich-server]           |

system info:

 OS: postmarketOS v25.06 aarch64
 Host: Xiaomi Mi 9T
 Kernel: Linux 6.15.0-rc2-sm7150
 Uptime: 3 days, 14 hours, 41 mins
 Packages: 625 (apk)
 Shell: bash 5.2.37

tribhuwan-kumar avatar Dec 16 '25 02:12 tribhuwan-kumar