buildx
buildx copied to clipboard
docker
Buildx doesn't respect insecure registries
Contributing guidelines
- [X] I've read the contributing guidelines and wholeheartedly agree
I've found a bug and checked that ...
- [X] ... the documentation does not mention anything about my problem
- [X] ... there are no open or closed issues that are related to my problem
Description
Using the buildx plugin the insecure registries are not working after the latest update. By default it tries HTTPS (443). Issue visible on multiple systems.
Expected behaviour
Use port 80
Actual behaviour
Uses port 443
Buildx version
github.com/docker/buildx v0.12.1 30feaa1
Docker info
Client: Docker Engine - Community
Version: 25.0.1
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.12.1
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.24.2
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 25.0.1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: a1496014c916f9e62104b33d1bb5bd03b0858e59
runc version: v1.1.11-0-g4bccb38
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 5.15.133.1-microsoft-standard-WSL2
Operating System: Ubuntu 22.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 11.58GiB
Name: Re4zOon-ThinkBook
ID: WYZY:FVFE:G4KA:VOCH:RMQ7:LJSQ:FEDQ:BUQ2:BR63:KVHY:WAFU:C2XH
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
our-registry
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
Builders list
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
default * docker
default default running v0.12.4+3b6880d2a00f linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/386
desktop-linux error
Cannot load builder desktop-linux: protocol not available
Configuration
FROM our-registry/base-images/debian:latest
RUN echo "hello"
Build logs
❯ docker build .
[+] Building 0.1s (2/2) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 112B 0.0s
=> ERROR [internal] load metadata for our-registry/base-images/debian:latest 0.0s
------
> [internal] load metadata for our-registry/base-images/debian:latest:
------
Dockerfile:1
--------------------
1 | >>> FROM our-registry/base-images/debian:latest
2 |
3 | RUN echo "hello"
--------------------
ERROR: failed to solve: our-registry/base-images/debian:latest: failed to do request: Head "https://our-registry/v2/base-images/debian/manifests/latest": dial tcp 192.168.1.1:443: connect: no route to host
Additional info
Insecure registry is set properly in /etc/docker/daemon.json:
❯ cat /etc/docker/daemon.json
{
"insecure-registries":["http://our-registry", "our-registry"],
"hosts": ["unix:///mnt/wsl/shared-docker/docker.sock"]
}
The actual error message you're seeing doesn't seem to be an HTTPS error:
ailed to solve: our-registrybase-images/debian:latest: failed to do request: Head "https://our-registry/v2/base-images/debian/manifests/latest": dial tcp 192.168.1.1:443: connect: no route to host
It looks like somehow it doesn't connect to your registry at all? Can you ping our-registry from your host without issue?
The problem is its trying to use HTTPS, even tho its in insecure registries. There was a similar (if not the same) issue with buildx some months ago: https://github.com/docker/buildx/issues/1642. Also using the old builder it works fine:
❯ export DOCKER_BUILDKIT=0 && export COMPOSE_DOCKER_CLI_BUILD=0
❯ docker build .
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
BuildKit is currently disabled; enable it by removing the DOCKER_BUILDKIT=0
environment-variable.
Sending build context to Docker daemon 2.048kB
Step 1/2 : FROM our-registry/base-images/debian:latest
latest: Pulling from base-images/debian
2f44b7a888fa: Pull complete
2af9ac79606f: Pull complete
a3ed95caeb02: Pull complete
50c9649d4a99: Pull complete
0fd1aca8d474: Pull complete
Digest: sha256:77b2ed0229b60e6f6599a178afd2e79a6488d2ca9db9db70cc312e7367f04b83
Status: Downloaded newer image for our-registry/base-images/debian:latest
---> 496d45f55202
Step 2/2 : RUN echo "hello"
---> Running in 8b159c428675
hello
---> Removed intermediate container 8b159c428675
---> 9afd9c6b8296
Successfully built 9afd9c6b8296
The problem is its trying to use HTTPS, even tho its in insecure registries.
Sure, that's possible there could be a bug here, but that's not what the error you've shared shows. #1642 is about http: server gave HTTP response to HTTPS client.
I'm unsure as to why the resolution for images should be different here, any ideas @crazy-max @thaJeztah? Given this is buildkit-in-docker.
Indeed, its not exactly the same. In any case, just try to build an image using a base from an insecure registry and you shall receive the same issue.
I think I saw a discussion somewhere where BuildKit initialises a client with either http or https, and because of that would not work with combinations (registry itself using http, but blobs downloaded through https and/or vice-versa). I wonder if that's related here.
cc @vvoland who did some work on this recently as well:
- https://github.com/moby/buildkit/pull/4293
- https://github.com/moby/buildkit/pull/4299
Related discussion for my previous comment;
- https://github.com/moby/moby/issues/47240#issuecomment-1914097057
Don't you need a toml file to configure builder instances to make insecure repos work with buildkit and buildx?
docker buildx create --name custom_builder --config conf.tomll
And conf.toml file like:
[registry."example.registry.com"]
insecure = true
Or am I missing the point here? That's what I've been doing for contacting insecure registries for over a year now. The setting in daemon.json don't affect the buildkit builder instances. You have to explicitly set them up.
Well, it worked until 25.x.y with buildx (which I guess is built on top of buildkit).
If you run docker build . on 24.x.y it works even without a toml config as long as the daemon is correctly set.
Oh yeah. By default it is buildx. We did not modify docker in any way (apart from the insecure registries in daemon.json). Its simple docker-ce from the repo.
[root@jenkins~]# docker build
ERROR: "docker buildx build" requires exactly 1 argument.
See 'docker buildx build --help'.
Usage: docker buildx build [OPTIONS] PATH | URL | -
Start a build
[root@jenkins~]#
Hi,
Any updates? Its blocking our second upgrade schedule for our jenkins'.
v0.13.1 fail use http registry. My flow shown as below
- docker buildx create --use --config buildx-config.toml
- docker buildx build --build-arg RELEASE=$RELEASE --build-arg DATE=$DATE --push --platform linux/amd64 . -t "$CI_REGISTRY_IMAGE:$RELEASE" -t "$CI_REGISTRY_IMAGE:$RELEASE-$DATE" -f Dockerfile
buildx-config.toml
[registry."192.168.0.2:5050"]
insecure = true
http = true
Buildx keep requesting https, in the end locking to old version with --driver-opt image=moby/buildkit:v0.12.2.
Is there any update with this issue ? we are being blocked by this exact issue and currently only working with the workaround --driver-opt image=moby/buildkit:v0.12.2
