Error "Temporary failure in name resolution" when resolving hosts

[ArtemAvramenko]

Description

After upgrading to 4.25.x, some hosts are no longer resolved from within containers. The type of container does not matter. No such problem on previous versions. Reproduced on both Windows 10 and Windows 11,

Reproduce

1. Open the terminal of any container
2. Install iputils-ping:

apt update
apt install iputils-ping

3. Execute ping:

ping login.microsoft.com

Expected behavior

The domain is expected to be resolved in the same way as in the host machine or in the WSL

docker version

Client:
 Cloud integration: v1.0.35+desktop.5
 Version:           24.0.6
 API version:       1.43
 Go version:        go1.20.7
 Git commit:        ed223bc
 Built:             Mon Sep  4 12:32:48 2023
 OS/Arch:           windows/amd64
 Context:           default

Server: Docker Desktop 4.25.2 (129061)
 Engine:
  Version:          24.0.6
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.7
  Git commit:       1a79695
  Built:            Mon Sep  4 12:32:16 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.22
  GitCommit:        8165feabfdfe38c65b599c4993d227328c231fca
 runc:
  Version:          1.1.8
  GitCommit:        v1.1.8-0-g82f18fe
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    24.0.6
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.5
    Path:     C:\Program Files\Docker\cli-plugins\docker-buildx.exe
  compose: Docker Compose (Docker Inc.)
    Version:  v2.23.0-desktop.1
    Path:     C:\Program Files\Docker\cli-plugins\docker-compose.exe
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-dev.exe
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     C:\Program Files\Docker\cli-plugins\docker-extension.exe
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.9
    Path:     C:\Program Files\Docker\cli-plugins\docker-init.exe
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-sbom.exe
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     C:\Program Files\Docker\cli-plugins\docker-scan.exe
  scout: Docker Scout (Docker Inc.)
    Version:  v1.0.9
    Path:     C:\Program Files\Docker\cli-plugins\docker-scout.exe

Server:
 Containers: 60
  Running: 33
  Paused: 0
  Stopped: 27
 Images: 18
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
 Kernel Version: 5.15.133.1-microsoft-standard-WSL2
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 9.718GiB
 Name: DESKTOP-BU0D40G
 ID: 041ff7fb-4c55-44f4-8a79-7954b9389887
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  tmetric-docker.datasoft.local
  127.0.0.0/8
 Live Restore Enabled: false

Diagnostics ID

27DFEF0D-70D2-4416-88FF-1C54CD2D0451/20231124112123

Additional Info

No response

[matt-coad-rp]

Spent a few days chasing this one.

If you check CoreDNS logs it reports buffer overflows whenever you attempt to resolve one of the dns names with the issue.

If from your container shell you forward the request to another DNS server. IE:

host login.microsoft.com 8.8.8.8

The request works.

Any attempts to resolve requests directly within your WSL host all work.

If I switched my workstations DNS servers to a server that was successful (I.E. 8.8.8.8) that made no difference. However, if I configured CoreDNS within kubernetes to forward requests to 8.8.8.8 then all requests would work. (Wasn't a solution as it breaks host.docker.internal)

[smshapiro85]

Following. Same issue on docker v. 4.25.2 on MacOS. Going to roll back to 4.24 for now.

Edit - Rolling back to 4.24 confirmed to resolve the DNS issue. Radarr/Sonarr in docker were unable to reach indexers, and my portainer agent was unable to communicate either on v. 4.25.2 but can now on 4.24.

[JoelBirlingmairBeastCode]

We have encountered the same issue and are going to roll back to 4.24 for now as a temporary workaround.

[ArtemAvramenko]

Does it make sense to upgrade to version 4.26.x? Or is the bug still not fixed?

[MihaelaStoica]

This seems to be related to https://github.com/docker/for-mac/issues/7110 and https://github.com/docker/for-mac/issues/7088

[JoelBirlingmairBeastCode]

Does it make sense to upgrade to version 4.26.x? Or is the bug still not fixed?

We have encountered the same issue on 4.26

[im-aIex]

Looks the root cause of this was fixed upstream with CoreDNS v1.11.2.

[im-aIex]

Still seeing this issue with 4.28.

[ArtemAvramenko]

The force_tcp option seems to help (see https://github.com/docker/for-mac/issues/7088#issuecomment-1880516515), but this workaround has not been fully tested yet.

To set this option you need to open the editor with the command:

kubectl edit configmap coredns -n kube-system -o yaml

After that add the force_tcp line, save the file and close the editor:

        ...
        forward . /etc/resolv.conf {
           force_tcp
           max_concurrent 1000
        }
        ...