docker
[BUG] Docker Compose Fails with 'exit status 128' When Including Remote Git Repository
Description
When using Docker Compose with the COMPOSE_EXPERIMENTAL_GIT_REMOTE feature to include a remote Git repository, the process fails with exit status 128.
Expected Behavior:
Docker Compose should successfully include the remote Git repository and start the defined services without errors.
Observed Behavior:
The process fails with the following debug output:
DEBU[0000] Enabled Docker Desktop integration (experimental) @ unix:///var/run/docker-cli.sock
DEBU[0000] otel error error="<nil>"
DEBU[0000] otel error error="<nil>"
exit status 128
Steps To Reproduce
Steps to Reproduce
-
Create a
docker-compose.ymlfile with the following content:services: hello_world: image: hello-world include: - [email protected]:conduktor/kafka-stack-docker-compose.git#master:zk-single-kafka-single.yml -
Execute the following command
GIT_TRACE=2 GIT_CURL_VERBOSE=2 GIT_TRACE_PERFORMANCE=2 GIT_TRACE_PACK_ACCESS=2 GIT_TRACE_PACKET=2 GIT_TRACE_PACKFILE=2 GIT_TRACE_SETUP=2 GIT_TRACE_SHALLOW=2 COMPOSE_EXPERIMENTAL_GIT_REMOTE=1 docker-compose --debug -f docker-compose.yml up -d
Compose Version
Docker Compose version v2.34.0-desktop.1
Docker Environment
Client:
Version: 28.0.4
Context: default
Debug Mode: false
Plugins:
ai: Docker AI Agent - Ask Gordon (Docker Inc.)
Version: v1.1.3
Path: /usr/local/lib/docker/cli-plugins/docker-ai
buildx: Docker Buildx (Docker Inc.)
Version: v0.22.0-desktop.1
Path: /usr/local/lib/docker/cli-plugins/docker-buildx
cloud: Docker Cloud (Docker Inc.)
Version: 0.2.20
Path: /usr/local/lib/docker/cli-plugins/docker-cloud
compose: Docker Compose (Docker Inc.)
Version: v2.34.0-desktop.1
Path: /usr/local/lib/docker/cli-plugins/docker-compose
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.38
Path: /usr/local/lib/docker/cli-plugins/docker-debug
desktop: Docker Desktop commands (Beta) (Docker Inc.)
Version: v0.1.6
Path: /usr/local/lib/docker/cli-plugins/docker-desktop
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.2
Path: /usr/local/lib/docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.27
Path: /usr/local/lib/docker/cli-plugins/docker-extension
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.4.0
Path: /usr/local/lib/docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /usr/local/lib/docker/cli-plugins/docker-sbom
scout: Docker Scout (Docker Inc.)
Version: v1.17.0
Path: /usr/local/lib/docker/cli-plugins/docker-scout
WARNING: Plugin "/usr/local/lib/docker/cli-plugins/docker-feedback" is not valid: failed to fetch metadata: fork/exec /usr/local/lib/docker/cli-plugins/docker-feedback: no such file or directory
WARNING: Plugin "/usr/local/lib/docker/cli-plugins/docker-scan" is not valid: failed to fetch metadata: fork/exec /usr/local/lib/docker/cli-plugins/docker-scan: no such file or directory
Server:
Containers: 88
Running: 1
Paused: 0
Stopped: 87
Images: 85
Server Version: 28.0.4
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
CDI spec directories:
/etc/cdi
/var/run/cdi
Swarm: inactive
Runtimes: nvidia runc io.containerd.runc.v2
Default Runtime: runc
Init Binary: docker-init
containerd version: 753481ec61c7c8955a23d6ff7bc8e4daed455734
runc version: v1.2.5-0-g59923ef
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
Kernel Version: 5.15.167.4-microsoft-standard-WSL2
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 7.761GiB
Name: docker-desktop
ID: 45a06112-7908-45bc-a705-f67815c99a48
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=unix:///var/run/docker-cli.sock
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
::1/128
127.0.0.0/8
Live Restore Enabled: false
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
WARNING: DOCKER_INSECURE_NO_IPTABLES_RAW is set
WARNING: daemon is not using the default seccomp profile
Anything else?
No response
We just introduced some additional diagnostic for git remote resources.
Please clone this repo, run make install to get compose built from sources, then run your command again with --verbose flag to get git command output. It might be necessary to cleanup local cache (%LOCALAPPDATA%/cache/docker-compose or %USER%/AppData/Local/cache/docker-compose)
Thank you for your prompt response and for providing additional diagnostics to address the issue. Unfortunately, after following your recommendations, the problem persists.
Steps Taken:
- Cloned the Repository and Built Compose from Source:
git clone https://github.com/docker/compose.git
cd compose
git rev-parse HEAD
c83f1285a88f5b772b32969285ad9e0b6c6867db
make
GO111MODULE=on go build -trimpath -tags "e2e" -ldflags "-w -X
github.com/docker/compose/v2/internal.Version=v2.35.0-2-gc83f1285a.m" -o "./bin/build/docker-compose" ./cmd
go: downloading github.com/compose-spec/compose-go/v2 v2.6.0
go: downloading github.com/moby/sys/atomicwriter v0.1.0
sudo cp bin/build/docker-compose /usr/local/bin/docker-compose
-
Cleared Local Cache
-
Verified Docker Compose Version:
docker-compose version
Docker Compose version v2.35.0-2-gc83f1285a.m
- Executed the docker-compose up Command with Verbose and Debug Flags:
COMPOSE_EXPERIMENTAL_GIT_REMOTE=true docker-compose --verbose -f app/docker-compose.yml up
DEBU[0000] Enabled Docker Desktop integration (experimental) @ unix:///var/run/docker-cli.sock
DEBU[0000] otel error error="<nil>"
DEBU[0000] otel error error="<nil>"
exit status 128
Despite enabling detailed Git tracing and debugging, the output does not provide additional insights into the root cause of the failure.
Hello @grzegorzb1990
I tried to reproduce, but without experiencing the same behaviour (on Mac first, I'll try on a Windows machin soon)
~/tmp/issue-12714 > bat --decorations=never compose.yaml 11:29:23
services:
hello_world:
image: hello-world
include:
- [email protected]:conduktor/kafka-stack-docker-compose.git#master:zk-single-kafka-single.yml
~/tmp/issue-12714 > GIT_TRACE=2 GIT_CURL_VERBOSE=2 GIT_TRACE_PERFORMANCE=2 GIT_TRACE_PACK_ACCESS=2 GIT_TRACE_PACKET=2 GIT_TRACE_PACKFILE=2 GIT_TRACE_SETUP=2 GIT_TRACE_SHALLOW=2 COMPOSE_EXPERIMENTAL_GIT_REMOTE=1 docker compose up 11:29:28
[+] Running 19/21
✔ hello_world Pulled 12.9s
✔ c9c5fd25a1bd Pull complete 9.9s
✔ zoo1 Pulled 38.9s
✔ 9351faf05db1 Pull complete 7.5s
✔ 7e732d40e412 Pull complete 37.3s
✔ kafka1 Pulled 38.8s
✔ 53b4abd6ab25 Pull complete 7.6s
⠼ a86352c4b8c9 Extracting 5 s 37.4s
✔ cfdb71e16ce9 Pull complete 7.6s
✔ 56f6649379c4 Pull complete 37.3s
✔ 72814fd5e773 Pull complete 7.6s
✔ 1d6620ad2620 Pull complete 7.6s
✔ 28c50053fee4 Pull complete 7.6s
✔ 58ab7c52b6eb Pull complete 7.6s
✔ 06bcf4799fa0 Pull complete 7.6s
✔ a9b3152af175 Pull complete 7.6s
✔ 37c3c8e73856 Pull complete 36.6s
✔ c1d0409f073a Pull complete 7.6s
⠼ 5d8104095c0c Extracting 1 s 37.4s
✔ dccb7daaa261 Pull complete 7.6s
✔ 3d241e90165b Pull complete 7.6s
[+] Running 4/4
✔ Network issue-12714_default Created 0.0s
✔ Container zoo1 Created 0.4s
✔ Container issue-12714-hello_world-1 Created 0.4s
✔ Container kafka1 Created 0.1s
Attaching to hello_world-1, kafka1, zoo1
hello_world-1 |
hello_world-1 | Hello from Docker!
hello_world-1 | This message shows that your installation appears to be working correctly.
hello_world-1 |
hello_world-1 | To generate this message, Docker took the following steps:
hello_world-1 | 1. The Docker client contacted the Docker daemon.
hello_world-1 | 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
hello_world-1 | (arm64v8)
hello_world-1 | 3. The Docker daemon created a new container from that image which runs the
hello_world-1 | executable that produces the output you are currently reading.
hello_world-1 | 4. The Docker daemon streamed that output to the Docker client, which sent it
hello_world-1 | to your terminal.
hello_world-1 |
hello_world-1 | To try something more ambitious, you can run an Ubuntu container with:
hello_world-1 | $ docker run -it ubuntu bash
hello_world-1 |
zoo1 | ===> User
hello_world-1 | Share images, automate workflows, and more with a free Docker ID:
zoo1 | uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
hello_world-1 | https://hub.docker.com/
zoo1 | ===> Configuring ...
hello_world-1 |
hello_world-1 | For more examples and ideas, visit:
hello_world-1 | https://docs.docker.com/get-started/
hello_world-1 |
hello_world-1 exited with code 0
kafka1 | ===> User
kafka1 | uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
kafka1 | ===> Configuring ...
kafka1 | Running in Zookeeper mode...
zoo1 | ===> Running preflight checks ...
zoo1 | ===> Check if /var/lib/zookeeper/data is writable ...
kafka1 | ===> Running preflight checks ...
kafka1 | ===> Check if /var/lib/kafka/data is writable ...
zoo1 | ===> Check if /var/lib/zookeeper/log is writable ...
kafka1 | ===> Check if Zookeeper is healthy ...
zoo1 | ===> Launching ...
zoo1 | ===> Printing /var/lib/zookeeper/data/myid
zoo1 | 1===> Launching zookeeper ...
kafka1 | [2025-04-14 09:30:15,620] INFO Client environment:zookeeper.version=3.8.4-9316c2a7a97e1666d8f4593f34dd6fc36ecc436c, built on 2024-02-12 22:16 UTC (org.apache.zookeeper.ZooKeeper)
kafka1 | [2025-04-14 09:30:15,620] INFO Client environment:host.name=kafka1 (org.apache.zookeeper.ZooKeeper)
kafka1 | [2025-04-14 09:30:15,620] INFO Client environment:java.version=17.0.12 (org.apache.zookeeper.ZooKeeper)
kafka1 | [2025-04-14 09:30:15,620] INFO Client environment:java.vendor=Eclipse Adoptium (org.apache.zookeeper.ZooKeeper)
kafka1 | [2025-04-14 09:30:15,620] INFO Client environment:java.home=/usr/lib/jvm/temurin-17-jre (org.apache.zookeeper.ZooKeeper)
...
kafka1 | [2025-04-14 09:30:17,812] INFO Kafka startTimeMs: 1744623017810 (org.apache.kafka.common.utils.AppInfoParser)
kafka1 | [2025-04-14 09:30:17,813] INFO [KafkaServer id=1] started (kafka.server.KafkaServer)
kafka1 | [2025-04-14 09:30:17,859] INFO [RequestSendThread controllerId=1] Controller 1 connected to kafka1:19092 (id: 1 rack: null) for sending state change requests (kafka.controller.RequestSendThread)
kafka1 | [2025-04-14 09:30:17,940] INFO [zk-broker-1-to-controller-forwarding-channel-manager]: Recorded new ZK controller, from now on will use node kafka1:19092 (id: 1 rack: null) (kafka.server.NodeToControllerRequestThread)
kafka1 | [2025-04-14 09:30:17,964] INFO [zk-broker-1-to-controller-alter-partition-channel-manager]: Recorded new ZK controller, from now on will use node kafka1:19092 (id: 1 rack: null) (kafka.server.NodeToControllerRequestThread)
kafka1 | [2025-04-14 09:30:22,761] INFO [Controller id=1] Processing automatic preferred replica leader election (kafka.controller.KafkaController)
Can you check the output of GIT_TRACE=2 GIT_CURL_VERBOSE=2 GIT_TRACE_PERFORMANCE=2 GIT_TRACE_PACK_ACCESS=2 GIT_TRACE_PACKET=2 GIT_TRACE_PACKFILE=2 GIT_TRACE_SETUP=2 GIT_TRACE_SHALLOW=2 COMPOSE_EXPERIMENTAL_GIT_REMOTE=1 docker-compose --debug -f docker-compose.yml config ?
Do you have the same issue if you're cloning the repository and run directly
GIT_TRACE=2 GIT_CURL_VERBOSE=2 GIT_TRACE_PERFORMANCE=2 GIT_TRACE_PACK_ACCESS=2 GIT_TRACE_PACKET=2 GIT_TRACE_PACKFILE=2 GIT_TRACE_SETUP=2 GIT_TRACE_SHALLOW=2 COMPOSE_EXPERIMENTAL_GIT_REMOTE=1 docker-compose --debug -f zk-single-kafka-single.yml up -d`
Can you also check Compose downloaded properly the project, as mentionned by ndeloof, here %LOCALAPPDATA%/cache/docker-compose or here %USER%/AppData/Local/cache/docker-compose
I had the same issue.
Changing git@... to https://... resolved the problem for me.
And, following this guide to set up my SSH key also worked.
Hello everybody,
we had the same problem. We were targeting a private repository and there are no ssh agents running on our machines.
The documentation states:
include allows you to reference Compose files from remote sources, such as OCI artifacts or Git repositories.
But it provides an example of the necessary syntax for oci:// only, unfortunately.
The problem is, that the git remote compose code transforms the string [email protected]#branch:pathinto a git command git ls-remote --exit-code [email protected] branch, which exits with 128 (as the host part is not valid). The next problem is, that the error message somehow disappears. You get a descriptive error message when debugging the code (directly after the execution), but this error message never gets printed to the caller's console.
The git ls-remote documentation describes all valid URL syntaxes (you get redirected to the git fetch documentation).
Long story short: For it to work properly using SSH, use this syntax:
include:
- ssh://[email protected]/$OWNER/$REPO.git#$BRANCH:$PATH
It would be great if anybody with more knowledge of the code could determine how the actual error message could be printed to the console. Furthermore, it might be a good idea to either add an SSH syntax example to the documentation or adapt the code in a way that the established syntax being used in the build-element ([email protected]/$OWNER/$REPO.git#$BRANCH:$PATH) can be used for includes as well.
Best regards, Manuel
@focdanisch thanks for sharing your workaround Sounds weird you get this error, as [email protected] is a valid SSH url (which is somehow flexible):
$ git ls-remote --exit-code [email protected]:docker/compose.git main
4d47da6dc2e42c28407d431a6a89cf8770eeeac9 refs/heads/main
anyway we should capture error reported by git command line and get it to the user console
Ah OK, further testing uncovered a subtle mistake I made : I used / instead of : in my URL.
[email protected]/$OWNER/$REPO.git#$BRANCH:$PATH => wrong
[email protected]:$OWNER/$REPO.git#$BRANCH:$PATH => works
ssh://[email protected]/$OWNER/$REPO.git#$BRANCH:$PATH => works as well
Thanks for the hint.
So, in my case, this is working on a private repository (I can access manually using ssh outside of compose):
include:
- path: [email protected]:$OWNER/$REPO.git#$BRANCH:$PATH
docker compose config returns the expected result. To recreate the problem, I change this to
include:
- path: https://github.com/$OWNER/$REPO.git#$BRANCH:$PATH
docker compose config just prints exit status 128 and nothing else. I guess, compose would ask for username/password, but the console is deactivated.