docker
Self signed certificate cannot be authenticated
Contributing guidelines
- [X] I've read the contributing guidelines and wholeheartedly agree
I've found a bug and checked that ...
- [ ] ... the documentation does not mention anything about my problem
- [X] ... there are no open or closed issues that are related to my problem
Description
ERROR: failed to solve: failed to push www.harbor.com/king/taxi-bus-server:0.1.9: failed to authorize: failed to fetch oauth token: Post "https://www.harbor.com/service/token": tls: failed to verify certificate: x509: certificate signed by unknown authority
Expected behaviour
Can be pushed normally
Actual behaviour
I searched for historical discussions and feedback on issues, but I still couldn't find a solution. I configured Toml and copied the certificate into the container. But x509 will still appear, but I can push normally using Docker Push
Buildx version
github.com/docker/buildx v0.10.5 86bdced
Docker info
Client: Docker Engine - Community
Version: 24.0.2
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.10.5
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.18.1
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 10
Running: 10
Paused: 0
Stopped: 0
Images: 19
Server Version: 24.0.2
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
runc version: v1.1.7-0-g860f061
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 3.10.0-1160.90.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.637GiB
Name: harbor
ID: f21487ee-68a3-48d6-997c-1df4e3577f87
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
www.harbor.com
127.0.0.0/8
Registry Mirrors:
https://3ighvcgt.mirror.aliyuncs.com/
Live Restore Enabled: false
Builders list
kingbuilder * docker-container
kingbuilder0 unix:///var/run/docker.sock running 81cd697 linux/amd64, linux/386
default docker
default default running v0.11.7-0.20230525183624-798ad6b0ce9f linux/amd64, linux/386
Configuration
FROM azul/zulu-openjdk:8-jre-headless-latest
RUN mkdir -p /app WORKDIR /app COPY ./target/yudao-server.jar app.jar ENV TZ=Asia/Shanghai ENTRYPOINT ["java","-jar","app.jar"]
docker buildx build --platform linux/amd64,linux/arm64 -t www.harbor.com/king/taxi-bus-server:0.1.9 --push .
Build logs
[+] Building 4.8s (17/17) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 583B 0.0s
=> [linux/amd64 internal] load metadata for docker.io/azul/zulu-openjdk:8-jre-headless-latest 1.8s
=> [linux/arm64 internal] load metadata for docker.io/azul/zulu-openjdk:8-jre-headless-latest 1.9s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [linux/amd64 1/4] FROM docker.io/azul/zulu-openjdk:8-jre-headless-latest@sha256:d73a99fee1fa81ba16f6fb37998be6204d7e9dac81c4e2cc0b645c2820d37cca 0.3s
=> => resolve docker.io/azul/zulu-openjdk:8-jre-headless-latest@sha256:d73a99fee1fa81ba16f6fb37998be6204d7e9dac81c4e2cc0b645c2820d37cca 0.3s
=> [internal] load build context 0.1s
=> => transferring context: 179B 0.0s
=> [linux/arm64 1/4] FROM docker.io/azul/zulu-openjdk:8-jre-headless-latest@sha256:d73a99fee1fa81ba16f6fb37998be6204d7e9dac81c4e2cc0b645c2820d37cca 0.3s
=> => resolve docker.io/azul/zulu-openjdk:8-jre-headless-latest@sha256:d73a99fee1fa81ba16f6fb37998be6204d7e9dac81c4e2cc0b645c2820d37cca 0.3s
=> CACHED [linux/amd64 2/4] RUN mkdir -p /app 0.0s
=> CACHED [linux/amd64 3/4] WORKDIR /app 0.0s
=> CACHED [linux/amd64 4/4] COPY ./target/yudao-server.jar app.jar 0.0s
=> CACHED [linux/arm64 2/4] RUN mkdir -p /app 0.0s
=> CACHED [linux/arm64 3/4] WORKDIR /app 0.0s
=> CACHED [linux/arm64 4/4] COPY ./target/yudao-server.jar app.jar 0.0s
=> ERROR exporting to image 1.6s
=> => exporting layers 0.0s
=> => exporting manifest sha256:24986b7a9a2a90be027deacd9af996e85a0918638f4af9c047e4ab4173b63c6f 0.1s
=> => exporting config sha256:e2550a7cbbc72a19c26d151ef15b01e081870fd0d2d12c3dcd99df44319a6330 0.1s
=> => exporting attestation manifest sha256:b45acfa2ff21ee3949e38051bd71665b6c82f5237252b9700494ece5cf559954 0.3s
=> => exporting manifest sha256:48093dce8b9e6d95234a7955036a743315d76fe599d3ed229804036ec71ead32 0.1s
=> => exporting config sha256:afdf5369c4787d7a11563750094f2e9a9ef768e0b3c9ca46c44cb69c713489c4 0.1s
=> => exporting attestation manifest sha256:c9629a393c4c893ec8255dd33a528e905cfe1245f88c735a585140b8cb2d31d9 0.2s
=> => exporting manifest list sha256:85244e7cd337ebd6f533946e6d59f830ddf7a0633fa02a1d4b91befb1288790b 0.1s
=> => pushing layers 0.0s
=> [auth] king/taxi-bus-server:pull,push token for www.harbor.com 0.0s
=> [auth] king/taxi-bus-server:pull,push token for www.harbor.com 0.0s
=> [auth] king/taxi-bus-server:pull,push token for www.harbor.com 0.0s
------
> exporting to image:
------
ERROR: failed to solve: failed to push www.harbor.com/king/taxi-bus-server:0.1.9: failed to authorize: failed to fetch oauth token: Post "https://www.harbor.com/service/token": tls: failed to verify certificate: x509: certificate signed by unknown authority
Additional info
debug = true
insecure-entitlements = [ "network.host", "security.insecure" ]
[registry."www.harbor.com"]
mirrors = [
"3ighvcgt.mirror.aliyuncs.com"
]
insecure = true
ca=["/etc/buildkit/www.harbor.com.crt"]
[[registry."www.harbor.com".keypair]]
key="/etc/buildkit/www.harbor.com.key"
cert="/etc/buildkit/www.harbor.com.cert"
As you're using a container builder, can you post your BuildKit logs please? Can be seen with docker logs buildx_buildkit_kingbuilder0.
Hi, I'm building a multi-arch image using buildx on Apple Silicon M1.
I've already added my private registry domain name into docker configuration insecure-registries setting.
Mine builder log:
time="2023-11-15T00:20:39Z" level=info msg="auto snapshotter: using overlayfs"
time="2023-11-15T00:20:39Z" level=warning msg="using host network as the default"
time="2023-11-15T00:20:39Z" level=info msg="found worker \"io9cq40biuaowbgafw3fwlp83\", labels=map[org.mobyproject.buildkit.worker.executor:oci org.mobyproject.buildkit.worker.hostname:a1f1153736eb org.mobyproject.buildkit.worker.network:host org.mobyproject.buildkit.worker.oci.process-mode:sandbox org.mobyproject.buildkit.worker.selinux.enabled:false org.mobyproject.buildkit.worker.snapshotter:overlayfs], platforms=[linux/arm64 linux/riscv64 linux/ppc64le linux/s390x linux/386 linux/mips64 linux/arm/v7 linux/arm/v6]"
time="2023-11-15T00:20:39Z" level=warning msg="skipping containerd worker, as \"/run/containerd/containerd.sock\" does not exist"
time="2023-11-15T00:20:39Z" level=info msg="found 1 workers, default=\"io9cq40biuaowbgafw3fwlp83\""
time="2023-11-15T00:20:39Z" level=warning msg="currently, only the default worker can be used."
time="2023-11-15T00:20:39Z" level=info msg="running server on /run/buildkit/buildkitd.sock"
time="2023-11-15T00:20:58Z" level=error msg="/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Canceled desc = context canceled"
time="2023-11-15T00:28:36Z" level=warning msg="forcibly turning on oci-mediatype mode for attestations" spanID=1a4e4df378aec7ea traceID=100db93628abd9a5d4bffff468c3277f
time="2023-11-15T00:28:37Z" level=error msg="/moby.buildkit.v1.Control/Solve returned error: rpc error: code = Unknown desc = failed to push docker.vpn/omsv-blank-database:20231114: failed to do request: Head \"https://docker.vpn/v2/omsv-blank-database/blobs/sha256:0c64aeff28be01d1b097dd81ca75dfe3596cf6d1a91fb8018280955cf48718eb\": tls: failed to verify certificate: x509: certificate signed by unknown authority"
Hi,
I've a very similar situation. Same errors messages about certificates.
2 local registries over https by self-generated certs and insecures-registries set on every docker daemon.
I can pull/push without any problems.... but can't build with images from my local registries because certs can't be verified.
@GGbind You may need to make the local machine trust your self-signed certificate by executing the following command:
cp /path/to/your_ca.crt /etc/pki/ca-trust/source/anchors/extca.crt
update-ca-trust
我在ubuntu 20.04上执行下面的操作解决了该问题： echo -n | openssl s_client -showcerts -connect harbor.XXX.local:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' >> ./my_harbor.crt
cp my_harbor.crt /usr/local/share/ca-certificates/
update-ca-certificates
@Utkarsh-vishnoi I successfully pushed the mirror value private harbor on Ubuntu 22.04 TLS using the following method.
- Generate a ca certificate using openssl and issue a harbor certificate.
- change harbor configuration: update IP, add certificate
- add the ca certificate to the local certificate trust chain:
sudo cp ca.crt /usr/local/share/ca-certificates
sudo update-ca-certificates
- Create the
buildkitd.tomlfile with the following contents
debug = true
insecure-entitlements = [ "network.host", "security.insecure" ]
[ registry. "your_harbor_ip" ]
insecure = true
- Execute buildx build named:
docker buildx create --use --driver docker-container --name buildx_main --config . /buildkitd.toml > /dev/null 2>&1 || true
docker buildx build --platform linux/amd64,linux/arm64 --push -f Dockerfile -t {your_harbor_ip}/test:1.0.0 .
``
debug = true insecure-entitlements = [ "network.host", "security.insecure" ]
[ registry. "your_harbor_ip" ] insecure = true
如果对于https的harbor自签镜像仓库，这样配置好像无效，只能使用方法3
I am having the exact same problem and I couldn't make it work. Running update-ca-certificates is not an option for me, but even when I tried, it didn't work.
Relevant code - it's an open source project: https://github.com/humansoftware/example_self_hosted_saas_app/pull/7
When I inspect the builder, I see the right config:
Run docker buildx inspect --bootstrap
docker buildx inspect --bootstrap
shell: /usr/bin/bash -e {0}
...
File#certs/harbor.local/ca.crt:
> -----BEGIN CERTIFICATE-----
> MIICxzCCAa+gAwIBAgIRAPEcmYalLZhUgCmVkc/+DSIwDQYJKoZIhvcNAQELBQAw
...
> -----END CERTIFICATE-----
>
File#buildkitd.toml:
> debug = true
> insecure-entitlements = ["network.host", "security.insecure"]
>
> [registry]
>
> [registry."harbor.local"]
> ca = ["/etc/buildkit/certs/harbor.local/ca.crt"]
> insecure = true
>
But when cache tries to export:
> importing cache manifest from harbor.local/example-self-hosted-saas-app/example-self-hosted-saas-app:buildcache:
------
------
> exporting cache to registry:
------
WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
3 warnings found (use docker --debug to expand):
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 22)
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 13)
- LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 21)
ERROR: failed to build: failed to solve: failed to fetch oauth token: Post "https://harbor.local/service/token": tls: failed to verify certificate: x509: certificate signed by unknown authority
Error: Process completed with exit code 1.
I am having the exact same problem and I couldn't make it work. Running update-ca-certificates is not an option for me, but even when I tried, it didn't work.
Relevant code - it's an open source project: humansoftware/example_self_hosted_saas_app#7
When I inspect the builder, I see the right config:
Run docker buildx inspect --bootstrap docker buildx inspect --bootstrap shell: /usr/bin/bash -e {0} ... File#certs/harbor.local/ca.crt: > -----BEGIN CERTIFICATE----- > MIICxzCCAa+gAwIBAgIRAPEcmYalLZhUgCmVkc/+DSIwDQYJKoZIhvcNAQELBQAw ... > -----END CERTIFICATE----- > File#buildkitd.toml: > debug = true > insecure-entitlements = ["network.host", "security.insecure"] > > [registry] > > [registry."harbor.local"] > ca = ["/etc/buildkit/certs/harbor.local/ca.crt"] > insecure = true >But when cache tries to export:
> importing cache manifest from harbor.local/example-self-hosted-saas-app/example-self-hosted-saas-app:buildcache: ------ ------ > exporting cache to registry: ------ WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load 3 warnings found (use docker --debug to expand): - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 22) - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 13) - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 21) ERROR: failed to build: failed to solve: failed to fetch oauth token: Post "https://harbor.local/service/token": tls: failed to verify certificate: x509: certificate signed by unknown authority Error: Process completed with exit code 1.
Try adding the certificate to the certificate trust chain on your local machine. Here's how to do it on Ubuntu:
sudo cp ca.crt /usr/local/share/ca-certificates
sudo update-ca-certificates
This is not running on my local machine, it's running on a k3s cluster produced by https://github.com/humansoftware/self-host-saas-k3s Here is how I create runners: https://github.com/humansoftware/self-host-saas-k3s/blob/main/roles/github_actions/templates/runnerdeployment.yaml.j2#L24
In my previous attempts, I tried something like bellow:
- name: runner
image: summerwind/actions-runner:latest
volumeMounts:
- name: harbor-ca
mountPath: /etc/buildkit/certs/{{harbor_domain}}/ca.crt
subPath: ca.crt
# Mount to a location for update-ca-certificates
- name: harbor-ca
mountPath: /usr/local/share/ca-certificates/harbor-ca.crt
subPath: ca.crt
lifecycle:
postStart:
exec:
command:
- /bin/sh
- -c
- |
update-ca-certificates && su runner -c /usr/bin/entrypoint.sh
securityContext:
runAsUser: 0
So update-ca-certificates was being run on my runner (I checked), but it still didn't work. When buildx starts a new container, it copies the certificate from my runner to the container, as inspect shows that happened successfully, but then the command update-ca-certificates would have to run inside the created container, not inside my runner's one.
This is confirmed in the documentation: https://docs.docker.com/reference/cli/docker/buildx/create/#buildkitd-config
Note that if you create a docker-container builder and have specified certificates for registries in the buildkitd.toml configuration, the files will be copied into the container under /etc/buildkit/certs and configuration will be updated to reflect that.
Correct me if I am wrong, but isn't this a problem in the default image container spawed by buildx? I can see the following in the setup logs:
/usr/local/bin/docker buildx inspect --bootstrap --builder builder-841865e2-48d2-4dca-b55d-8625b8f6c9fe
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 4.9s done
#1 creating container buildx_buildkit_builder-841865e2-48d2-4dca-b55d-8625b8f6c9fe0
#1 creating container buildx_buildkit_builder-841865e2-48d2-4dca-b55d-8625b8f6c9fe0 2.1s done
#1 DONE 7.0s
Name: builder-841865e2-48d2-4dca-b55d-8625b8f6c9fe
Driver: docker-container
Last Activity: 2025-07-06 23:16:17 +0000 UTC
So moby/buildkit:buildx-stable-1 should somehow be using the CA certificate and it's not - maybe a bug in this image or this is the image that has to run update-ca-certificates ?