[Bug]: Docker compose fails to pull image when using Remote Docker Engine

[ebreton]

Is there an existing issue for this?

• [X] I have searched the existing issues

Example public repository

https://github.com/ebreton/coolify-issue

Description

When I create an application with the docker-compose template on a remote Docker destination, it fails to deploy.

I get the following error:

[16:41:15.600] Debug logging is disabled. Enable it above if necessary!
[16:41:15.635] Cloning ebreton/coolify-issue:main...
[16:41:16.706] Configuration changed
[16:41:20.351] Command failed with exit code 18: docker compose --project-directory /tmp/build-sources/ebreton/coolify-issue/cldhjufh4001qa2pmh9ah7da8 -f /tmp/build-sources/ebreton/coolify-issue/cldhjufh4001qa2pmh9ah7da8/docker-compose.yml pull
kibana Pulling 
elasticsearch Pulling 
time="2023-01-29T15:41:19Z" level=warning msg="commandConn.CloseWrite: commandconn: failed to wait: signal: killed"
elasticsearch Error 
kibana Error 
error during connect: Post "http://docker.example.com/v1.41/images/create?fromImage=elasticsearch&tag=8.6.0": command [ssh -- 162.55.162.246-remote docker system dial-stdio] has exited with signal: killed, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=
[16:41:20.358] Command failed with exit code 18: docker compose --project-directory /tmp/build-sources/ebreton/coolify-issue/cldhjufh4001qa2pmh9ah7da8 -f /tmp/build-sources/ebreton/coolify-issue/cldhjufh4001qa2pmh9ah7da8/docker-compose.yml pull
kibana Pulling 
elasticsearch Pulling 
time="2023-01-29T15:41:19Z" level=warning msg="commandConn.CloseWrite: commandconn: failed to wait: signal: killed"
elasticsearch Error 
kibana Error 
error during connect: Post "http://docker.example.com/v1.41/images/create?fromImage=elasticsearch&tag=8.6.0": command [ssh -- 162.55.162.246-remote docker system dial-stdio] has exited with signal: killed, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=

Whereas I was expecting the same output that I would have with a local docker engine

[16:20:35.086] Cloning ebreton/coolify-issue:main...
[16:20:39.605] Pulling images from Compose file...
[16:20:39.767] Building images from Compose file...
[16:20:41.519] Deployed 🎉

Steps To Reproduce

As a prerequisite, you need to set up a Remote Docker Destination

1. Click on Create New Resource
2. Choose Application
3. Fill https://github.com/ebreton/coolify-issue in Pulbic Repository from Git
4. Click on Load repository and select the main branch
5. Select your remote Docker destination
6. In Coolify Base, select the Docker Compose button
7. You do not need to set any specific configuration, secrets or else
8. click on deploy

=> you will have the traceback pasted above

If you repeat the same steps on your Local Docker destination, everything is fine

When tyring to find the solution, I came across the following:

• This answer in stack overflow points out that the weird domain docker.example.com comes from a placeholder domain name for connecting over SSH. Therefore more confusing than helpful
• This post suggests to set DOCKER_HOST system wise, but this did not change anything for me. And I did not see how the rootless context could be set on this case

Version

3.12.17

[otryshko]

This bug has been fixed in docker compose 2.16.0 (see https://github.com/docker/compose/issues/9448). coolify is locked to 2.6.1 https://github.com/coollabsio/coolify/blob/cc2f83c4d9d4b0425856a91f3c5dcd2be01a4c9d/Dockerfile#L22-L23

There are claims that it env extension problems (which made coolify to use 2.6.1) have been (partially?) fixed in 2.11.2 (I can't validate it since I have no context about it) https://github.com/docker/compose/issues/9806#issuecomment-1259842614

@andrasbacsai can we move to docker compose 2.16.0 ? I can open a PR to rev the version; is there a way to validate that it still works after the change? in the current setup remote docker engine is just flat out broken (works only for 1 service per docker-compose file if it is a linux-linux connection).

[andrasbacsai]

I'm closing this issue, because tracking features will be moved to GitHub discussions. Github issues will be only for bugs.

Version 4 is in full focus. v3 related issues will be closed, if it is not a critical bug.

Thanks for your understanding.