moby
using an external dockerfile frontend on windows slows down the build
when I add: # syntax=docker/dockerfile:1 at the start of my dockerfile one of the first things the build does is tries
=> resolve image config for docker.io/docker/dockerfile: 68.7s
which takes 68 seconds. It looks like it's trying to pull the latest version and maybe it just takes that long, but I'd like a way to improve this as it adds a 60 second penalty every build I try to run.
Windows version: 10.0.22621.2134 Docker Desktop: 4.22.1 Docker Engine: 24.0.5 Buildkit v0.11.6+0a15675913b7 Buildx: v0.10.4 c513d34049e499c53468deac6c4267ee72948f02
I am AFK untill the 18th, but I'll gladly have a look as soon as I get back.
Actually, this is probably not windows containers at all as docker/dockerfile is not released for windows atm. If this is linux with windows client then need to understand what is different in this specific system or a way to reproduce.
That's correct I'm not using Windows containers.
What would you like as a reproduction? I can setup a simple build and give you some verbose logs.
super simple build:
# syntax=docker/dockerfile:1
FROM node:20.5
CMD ["node", "index.js"]
output:
PS C:\clipboard> docker build .
[+] Building 99.1s (7/7) FINISHED
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 109B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 69.6s
=> CACHED docker-image://docker.io/docker/dockerfile:1@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50 0.0s
=> [internal] load metadata for docker.io/library/node:20.5 29.1s
=> CACHED [1/1] FROM docker.io/library/node:20.5@sha256:8d9887b3b05d2e65598a18616c37cfc271346d12248dfcbeadd7b7bf 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:bb4ae6999bc39e3f387c8940c2ce4dea7492df52475a0786cc19c01fc2f22c29 0.0s
What's Next?
View summary of image vulnerabilities and recommendations → docker scout quickview
the logs seem to indicate that the image config is cached but it doesn't seem like it, it still takes a while if I run it again:
PS C:\clipboard> docker build .
[+] Building 48.3s (7/7) FINISHED
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 109B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 22.5s
=> CACHED docker-image://docker.io/docker/dockerfile:1@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50 0.0s
=> [internal] load metadata for docker.io/library/node:20.5 25.5s
=> CACHED [1/1] FROM docker.io/library/node:20.5@sha256:8d9887b3b05d2e65598a18616c37cfc271346d12248dfcbeadd7b7bf 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:bb4ae6999bc39e3f387c8940c2ce4dea7492df52475a0786cc19c01fc2f22c29 0.0s
What's Next?
View summary of image vulnerabilities and recommendations → docker scout quickview
had a coworker try and his only took 3 seconds, so it's not anything local, seems like it's just my machine. Let me know what other diagnostics you want me to do.
See if you can reproduce this in container builder and if you can then get a trace with opentelemetry https://docs.docker.com/build/building/opentelemetry/
I just tried the following:
docker run -d --name jaeger -p "6831:6831/udp" -p "16686:16686" --restart unless-stopped jaegertracing/all-in-one- docker buildx create --use
--name mybuilder
--driver docker-container
--driver-opt "network=host"
--driver-opt "env.JAEGER_TRACE=localhost:6831" docker buildx build . --builder mybuilderdocker build .
I'm not super sure if step 3 was the right thing to do to use the builder but I think it was, however it only took 5 seconds to resolve the image config. I followed it up with a docker build . to verify that I was still having the issue outside a custom builder. Here's the full log from the console. I'm not sure what's in Jaeger would be helpful.
PS C:\clipboard\Docker testing> docker buildx build . --builder mybuilder
[+] Building 12.8s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 109B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 5.0s
=> docker-image://docker.io/docker/dockerfile:1@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 4.0s
=> => resolve docker.io/docker/dockerfile:1@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 0.0s
=> => sha256:9d9c93f4b00be908ab694a4df732570bced3b8a96b7515d70ff93402179ad232 11.80MB / 11.80MB 3.8s
=> => extracting sha256:9d9c93f4b00be908ab694a4df732570bced3b8a96b7515d70ff93402179ad232 0.1s
=> [internal] load metadata for docker.io/library/node:20.5 3.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [1/1] FROM docker.io/library/node:20.5@sha256:8d9887b3b05d2e65598a18616c37cfc271346d12248dfcbeadd7b7bf4da1e827 0.3s
=> => resolve docker.io/library/node:20.5@sha256:8d9887b3b05d2e65598a18616c37cfc271346d12248dfcbeadd7b7bf4da1e827 0.0s
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
PS C:\clipboard\Docker testing> docker build .
[+] Building 93.9s (7/7) FINISHED
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 109B 0.0s
=> resolve image config for docker.io/docker/dockerfile:1 69.5s
=> CACHED docker-image://docker.io/docker/dockerfile:1@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 0.0s
=> [internal] load metadata for docker.io/library/node:20.5 24.0s
=> CACHED [1/1] FROM docker.io/library/node:20.5@sha256:8d9887b3b05d2e65598a18616c37cfc271346d12248dfcbeadd7b7bf4da1e827 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:bb4ae6999bc39e3f387c8940c2ce4dea7492df52475a0786cc19c01fc2f22c29 0.0s
What's Next?
View summary of image vulnerabilities and recommendations → docker scout quickview
My guess is that something's up with the networking environment of the docker daemon, that doesn't affect the docker-container driver when using host-networking.
This could be a WSL2 issue of some kind... You should make sure you're on the Windows store releases of WSL2, and using the latest version (1.2.5.0 currently).
Have you tried docker build --network=host .? That should rule my theory in or out.
Some things I noticed;
- your initial report shows an older version of buildx being used (
v0.10.4), but I think Docker Desktop 4.22.1 ships with a v0.11.x version of buildx; was that a custom install of the buildx cli? - I initially was wondering if it was delay related to authentication (we've had some issues with the wincreds credential helper in that area), although I think BuildKit nowadays shows
[auth]in the progress output when it's building. - Are you seeing the same issue if you start the build from a WSL2 shell (not from a Windows host)?
@thaJeztah I tried it out in WSL and it took a while but I actually got an error which is new to me.
[+] Building 22.7s (3/3) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 109B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> ERROR resolve image config for docker.io/docker/dockerfile:1 22.5s
------
> resolve image config for docker.io/docker/dockerfile:1:
------
Dockerfile:1
--------------------
1 | >>> # syntax=docker/dockerfile:1
2 | FROM node:20.5
3 |
--------------------
ERROR: failed to solve: failed to do request: Head "https://docker.io.registry.lingnet/v2/docker/dockerfile/manifests/1?ns=docker.io": dialing docker.io.registry.lingnet:443 with direct connection: connecting to 10.1.1.135:443: dial tcp 10.1.1.135:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
this looks like our local docker mirror is down, that's what: docker.io.registry.lingnet is. After checking it looks like it's not online anymore. If I remove
"registry-mirrors": ["https://docker.io.registry.lingnet"]
from my docker engine config in the Docker Desktop GUI that made the issue go away. I'm sorry I totally forgot about that, and it looks like on windows that failure was getting ignored but when I ran it in wsl it actually throws an error.
For your question about buildx, no this isn't a custom install just a normal install from Docker Desktop I believe. It looks like there's 2 copies of buildx on my machine
- ~/.docker/cli-plugins/docker-buildx.exe
- C:\Program Files\Docker\Docker\resources\cli-plugins\docker-buildx.exe
the copy in my home directory is version v0.10.4 and the copy in program files is v0.11.2-desktop.4 how do I get docker to use the correct version?
A bit more investigation and it looks like WSL uses buildx version v0.11, but powershell uses buildx version v0.10
I've worked out my issue, but it would be nice if there was better feedback that the registry mirror was down instead of just failing silently.
how do I get docker to use the correct version?
~/.docker/cli-plugins/docker-buildx.exe takes precedence over the version shipped in Docker Desktop so you should remove ~/.docker/cli-plugins/docker-buildx.exe to avoid drifting.
@thaJeztah Not sure if we discussed about this but I wonder if we should warn users on DD that some CLI plugins are currently overriding ones shipped with DD?
Looking at this path ~/.docker/cli-plugins/docker-buildx.exe it seems inside your WSL distro. What about the host?
Can you show the output of the command docker info on WSL and Windows host please?
I tried it out in WSL and it took a while but I actually got an error which is new to me.
On Windows (powershell), can you show the output of the following commands:
(get-command docker-credential-desktop.exe).Path
(get-command docker-credential-wincred.exe).Path
docker-credential-desktop.exe version
docker-credential-wincred.exe version
Thanks I've deleted that plugin to stay in sync.
No sorry I understand that was a bit confusing. This is just on windows, so under %userprofile%\.docker\cli-plugins\docker-buildx.exe just my logged in user folder. But ~ also refers to %userprofile% on windows these days.
docker info windows:
Client:
Version: 24.0.6
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2-desktop.4
Path: C:\Program Files\Docker\cli-plugins\docker-buildx.exe
compose: Docker Compose (Docker Inc.)
Version: v2.11.1
Path: C:\Users\kevin\.docker\cli-plugins\docker-compose.exe
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.0
Path: C:\Program Files\Docker\cli-plugins\docker-dev.exe
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.20
Path: C:\Program Files\Docker\cli-plugins\docker-extension.exe
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v0.1.0-beta.7
Path: C:\Program Files\Docker\cli-plugins\docker-init.exe
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: C:\Program Files\Docker\cli-plugins\docker-sbom.exe
scan: Docker Scan (Docker Inc.)
Version: v0.26.0
Path: C:\Program Files\Docker\cli-plugins\docker-scan.exe
scout: Command line tool for Docker Scout (Docker Inc.)
Version: 0.24.1
Path: C:\Program Files\Docker\cli-plugins\docker-scout.exe
Server:
Containers: 78
Running: 72
Paused: 0
Stopped: 6
Images: 56
Server Version: 24.0.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
runc version: v1.1.8-0-g82f18fe
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
Kernel Version: 5.15.90.1-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 20
Total Memory: 15.47GiB
Name: docker-desktop
ID: 969c7f0a-46ef-4826-8275-d1a2ddf95a9a
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
WARNING: daemon is not using the default seccomp profile
docker info wsl
Client:
Version: 24.0.6
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.11.2-desktop.4
Path: C:\Program Files\Docker\cli-plugins\docker-buildx.exe
compose: Docker Compose (Docker Inc.)
Version: v2.11.1
Path: C:\Users\kevin\.docker\cli-plugins\docker-compose.exe
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.0
Path: C:\Program Files\Docker\cli-plugins\docker-dev.exe
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.20
Path: C:\Program Files\Docker\cli-plugins\docker-extension.exe
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v0.1.0-beta.7
Path: C:\Program Files\Docker\cli-plugins\docker-init.exe
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: C:\Program Files\Docker\cli-plugins\docker-sbom.exe
scan: Docker Scan (Docker Inc.)
Version: v0.26.0
Path: C:\Program Files\Docker\cli-plugins\docker-scan.exe
scout: Command line tool for Docker Scout (Docker Inc.)
Version: 0.24.1
Path: C:\Program Files\Docker\cli-plugins\docker-scout.exe
Server:
Containers: 78
Running: 72
Paused: 0
Stopped: 6
Images: 56
Server Version: 24.0.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
runc version: v1.1.8-0-g82f18fe
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
Kernel Version: 5.15.90.1-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 20
Total Memory: 15.47GiB
Name: docker-desktop
ID: 969c7f0a-46ef-4826-8275-d1a2ddf95a9a
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
WARNING: daemon is not using the default seccomp profile
credential helper info:
PS C:\> (get-command docker-credential-desktop.exe).Path
C:\Program Files\Docker\Docker\resources\bin\docker-credential-desktop.exe
PS C:\> (get-command docker-credential-wincred.exe).Path
C:\Program Files\Docker\Docker\resources\bin\docker-credential-wincred.exe
PS C:\> docker-credential-desktop.exe version
docker-credential-desktop (github.com/docker/docker-credential-helpers) v0.7.0
PS C:\> docker-credential-wincred.exe version
docker-credential-wincred (github.com/docker/docker-credential-helpers) v0.7.0
FYI since I opened this issue I updated to the latest version of docker desktop v4.23.0