docker compose v2.6.0 build error "http: server gave HTTP response to HTTPS client"

[Couga54]

When trying to build with docker compose v2.6.0 I got error:

[root@buildagent docker]# docker compose build
[+] Building 0.6s (4/4) FINISHED
 => [internal] load build definition from Server.Dockerfile                                                                                                                                                                   0.4s
 => => transferring dockerfile: 102B                                                                                                                                                                                                 0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                    0.5s
 => => transferring context: 91B                                                                                                                                                                                                     0.0s
 => ERROR [internal] load metadata for my.private.registry:5001/repo/docker-registry/server/runtime:1.0                                                                                                                    0.1s
 => [internal] load metadata for my.private.registry:5001/repo/docker-registry/server/deps:1.0                                                                                                                             0.0s
------
 > [internal] load metadata for my.private.registry:5001/repo/docker-registry/server/runtime:1.0:
------
failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to do request: Head "https://my.private.registry:5001/v2/repo/docker-registry/server/runtime/manifests/1.0": http: server gave HTTP response to HTTPS client

My private registry is added in daemon.json and build works perfectly when using docker-compose v1.29.2

[root@buildagent docker]# cat /etc/docker/daemon.json
{
  "insecure-registries" : [ "my.private.registry:5001" ]
}

docker info

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  compose: Docker Compose (Docker Inc., v2.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 25
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1127.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 1.795GiB
 Name: buildagent.h30-100
 ID: TCZ5:2DHF:SDOB:TWBH:3WTU:TYAF:ID4Z:7JEK:NRVQ:URP6:HLF5:XEU3
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  my.private.registry:5001
  127.0.0.0/8
 Live Restore Enabled: false

docker compose version

Docker Compose version v2.6.0

[pvcostescu]

I am also experiencing this issue with Compose v2.6.0. I have the insecure registries showing in docker info, and running the Dockerfile in question with regular docker build is able to pull the images from the insecure repository. but running the same Dockerfile from a compose file doesn't seem to respect the insecure registry setting.

[sunnyjocker]

docker buildx buid result the same. but docker pull my.private.registry:5000/foo/bar works.

[astroesteban]

I too am having that issue. When I run docker compose build my-service it tries to pull my image in the Dockerfile from the insecure registry, FROM my-private-image AS base and errors out with the http to https error.

I am running v2.9.0 on Mac OS.

So I've found a couple of ugly temporary fixes:

1. Instead of docker compose build run docker build.
2. Run docker pull my-private-image and then run docker compose build
3. Lastly running DOCKER_BUILDKIT=0 docker compose build my-service works as well but then I can't use things like build-time secrets.

Is there an official fix for this?

[kamikaze]

Same with Docker Desktop for Windows. It seems docker-compose doesn't respect the insecure-registries setting and has no option to force it...

[devnix]

Same problem here, using Docker Engine. Forced to run docker pull my-private-image and then docker compose build

[croensch]

As far as i know/remember docker-compose v1 was doing an HTTPS request first, and if your certificate were self-signed or otherwise not valid, would silently do an HTTP request second - instead of failing.

How do i know that? Well, Kubernetes would not do this fallback (it would do an HTTP request in the first place) and present the same or a very similar error. Our Nexus Repository Manager had :443 with a self-signed cert in the beginning but we moved to port :18443 which is a plain HTTP endpoint again. That was working in docker / docker-compose v1 / minikube

[ecederstrand]

Setting the protocol explicitly in daemon.json fixed this for us:

{
    "insecure-registries": [
        "http://my-registry.nexus.internal:1234"
    ]
}

[ndeloof]

please try reproducing this build failure using docker buildx build ... for your Dockerfile. If you get the same issue failing to pull base image, then this is a buildkit issue. Otherwise there's something wrong with the way compose setup buildx for the build...

[yrsh]

I have the same issue. And "insecure-registries" does not help, seems that compose just ignores it and always use https.
Docker Compose version v2.15.1

[ndeloof]

@yrsh can you please try building directly your Dockerfile with buildkit enabled ? (run docker buildx build ... )

[croensch]

I studied the change logs of Docker Desktop, Engine, Compose, etc and now understand that some flags have changed this behavior even for Compose V1.

Docker 23.0.1 + Docker Compose 1.29.2: DOCKER_BUILDKIT=0 works and DOCKER_BUILDKIT=1 breaks. Prefixing insecure-registries with http:// works with DOCKER_BUILDKIT=1.

All with COMPOSE_DOCKER_CLI_BUILD=1 forced, so it works like Compose V2.

The only documentation i found on this change is: https://github.com/docker/docs/pull/16578

[whatwewant]

docker/docs#16578

works for me insecure-registrie use http://ip:port instead. of ip:port

[dginhoux]

Hi,

I've tried to use http:// prefix but :

Mar 27 22:50:04 srv-swarm-worker3.infra.ginhoux.net dockerd[781229]: time="2023-03-27T22:50:04.481784426+02:00" level=warning msg="insecure registry http://registry-cache.infra.ginhoux.net:5000 should not contain 'http://' and 'http://' has been removed from the insecure registry config"

Mar 27 22:50:04 srv-swarm-worker3.infra.ginhoux.net dockerd[781229]: time="2023-03-27T22:50:04.481842066+02:00" level=warning msg="insecure registry http://registry-build.infra.ginhoux.net:5001 should not contain 'http://' and 'http://' has been removed from the insecure registry config"

So, it's not a good idea... (Docker version 23.0.1, build a5ee5b1)

Have a good day

[Couga54]

Looks like the issue is fixed in the latest versions: docker compose v2.17.3 docker 23.0.5 A private registry is added without the protocol:

{
    "insecure-registries" : [ "my.private.registry.com:5001" ]
}

Docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.4
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.17.3
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 1018
 Server Version: 23.0.5
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc version: v1.1.5-0-gf19387a
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 3.10.0-1160.88.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 3.7GiB
 Name: vm-buildagent
 ID: KHGR:EQC7:X3CU:W6FP:5VLB:5QVV:VT2A:D44P:5HYP:4ZC3:NZ62:IH7D
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Experimental: false
 Insecure Registries:
  my.private.registry.com:5001
  127.0.0.0/8
 Live Restore Enabled: false