docker
docker do not honor custom ssh port in ADD instruction while cloning git repositories
Description
I'm self-hosting git server. It only allows cloning repos via ssh. I've change default ssh port.
I'm trying to use ADD instruction to add my repository to docker image, but it seems that docker doesn't honor other ssh port.
FROM ubuntu:20.04
WORKDIR /project
ADD --keep-git-dir=false ssh://[email protected]:12345/org/project.git /project
Reproduce
docker build --ssh default --tag bug/reproduce:latest -f Dockerfile .
> [3/3] ADD --keep-git-dir=false ssh://[email protected]:12345/org/project.git /project
0.024 Initialized empty Git repository in /var/lib/docker/overlay2/72yquf1x6jb9lb8xkgjtaol2k/diff/
0.355 Host key verification failed.
0.357 fatal: Could not read from remote repository.
0.357
0.357 Please make sure you have the correct access rights
0.357 and the repository exists.
Expected behavior
I expect docker to honor custom ports.
docker version
Client: Docker Engine - Community
Version: 26.1.2
API version: 1.45
Go version: go1.21.10
Git commit: 211e74b
Built: Wed May 8 13:59:59 2024
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 26.1.2
API version: 1.45 (minimum version 1.24)
Go version: go1.21.10
Git commit: ef1912d
Built: Wed May 8 13:59:59 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.31
GitCommit: e377cd56a71523140ca6ae87e30244719194a521
runc:
Version: 1.1.12
GitCommit: v1.1.12-0-g51d5e94
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client: Docker Engine - Community
Version: 26.1.2
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.14.0
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.27.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 12
Running: 0
Paused: 0
Stopped: 12
Images: 26
Server Version: 26.1.2
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 nvidia runc
Default Runtime: runc
Init Binary: docker-init
containerd version: e377cd56a71523140ca6ae87e30244719194a521
runc version: v1.1.12-0-g51d5e94
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.5.0-28-generic
Operating System: Ubuntu 22.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 15.4GiB
Name: oleksandr-ROG-Zephyrus-M15-GU502LW
ID: f5e63e23-3a08-4d37-92a3-a76cb7aeb399
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional Info
Workaround: modifying ~/.ssh/config file with
Host example.com
HostName example.com
Port 12345
works, but this requires to modify ssh/config on every machine that may need to build this image
Sorry for this back and forth movements. I've discovered that my work arroung doesn't work, so I've hesitated about source of error. Still I'm not sure about where it is, so I've decided to move on and reopen the issue. Additional logs here:
> [5/6] ADD --keep-git-dir=true [email protected]:2222/repo.git /repo:
0.015 Initialized empty Git repository in /var/lib/docker/overlay2/ra7cdlohkb49maegh7vqsrgrc/diff/
1.061 Permission denied, please try again.
1.152 Permission denied, please try again.
1.244 Received disconnect from <HIDDEN IP> port 22:2: Too many authentication failures
1.244 Disconnected from <HIDDEN IP> port 22
1.246 fatal: Could not read from remote repository.
1.246
1.246 Please make sure you have the correct access rights
1.246 and the repository exists.
------
ERROR: failed to solve: failed to load cache key: error fetching default branch for repository [email protected]:2222/repo.git: git error: exit status 128
stderr:
Permission denied, please try again.
Permission denied, please try again.
Received disconnect from <HIDDEN IP> port 22:2: Too many authentication failures
Disconnected from <HIDDEN IP> port 22
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The underlying error is coming from BuildKit here; not sure if it indeed currently allows a custom port to be used, but let me transfer this ticket to the buildx issue tracker, which is the client used when running docker build.
This should have been fixed in https://github.com/moby/buildkit/pull/4142 (see https://github.com/moby/buildkit/pull/4069 for more context).
Not sure which version of buildkit is vendored in moby 26.1.2 though, so not sure if it includes that fix?
moby v26.1.1 has buildkit v0.13.2 as a dependency. v0.13.2 has its code updated with changes that you've mentioned as far as I can tell.
it seems I've found a fix. By adding
# syntax=docker/dockerfile:1
FROM ...
I've manage to use ADD directive with custom port in ssh url.
I'm wondering what is default value of syntax used anyway?
I think the default would be the "frontend" code that's part of the BuildKit codebase that's used (so in this case, the v0.13.2 code https://github.com/moby/buildkit/tree/v0.13.2).
That is, if you're using the default builder in the docker engine, but if you're using a custom builder (docker buildx create), you'd be running a BuildKit instance running in a container, so that one may be a different version.