Buildx support for rancher (and/or homebrew's) docker

[andrewstein]

Description

We are trying to use the new buildx option for the docker-maven-plugin -- thank you for implementing this.

The build command generated by the plugin is:

 docker --config <some-path>/docker buildx create --driver docker-container --name maven

On a machine with the docker supplied by the docker desktop everything seems to work fine. However, on a machine with the docker supplied by rancher I get the following:

[INFO] DOCKER> unknown flag: --driver
[INFO] DOCKER> See 'docker --help'.
...<usage info>

Running the docker command directly on the cli, produces a similar result. A colleague using the docker command from homebrew sees a similar error message.

Please update the buildx support for rancher's and/or homebrew's docker.

Info

• docker-maven-plugin version : 0.40.2, 0.41.0
• Maven version (mvn -v) :

Apache Maven 3.8.7 (b89d5959fcde851dcb1c8946a785a163f14e1e29)
Maven home: /opt/homebrew/Cellar/maven/3.8.7/libexec
Java version: 11.0.17, vendor: Homebrew, runtime: /opt/homebrew/Cellar/openjdk@11/11.0.17/libexec/openjdk.jdk/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "13.1", arch: "aarch64", family: "mac"

• Docker version :

$ docker --version
Docker version 20.10.21-rd, build ac29474

• Buildx version:

$ docker buildx version
github.com/docker/buildx v0.9.1 ed00243a0ce2a0aee75311b06e32d33b44729689

• If it's a bug, how to reproduce : Use the plugin with

                <buildx>
                  <platforms>
                    <platform>linux/amd64</platform>
                    <platform>linux/arm64</platform>
                  </platforms>
                </buildx>

on a machine with rancher's docker.

[andrewstein]

Some of my diagnostics: It seems that the --driver flag is not the issue here. Rather the first --config flag (to docker rather than to buildx) is causing the docker cli to fail.

[jflinchbaugh]

versions when seeing the failure with the client from homebrew and the server from rancher:

$ docker version
Client: Docker Engine - Community
 Version:           23.0.1
 API version:       1.41 (downgraded from 1.42)
 Go version:        go1.19.5
 Git commit:        a5ee5b1dfc
 Built:             Thu Feb  9 19:15:59 2023
 OS/Arch:           darwin/amd64
 Context:           default

Server:
 Engine:
  Version:          20.10.20
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       03df974ae9e6c219862907efdd76ec2e77ec930b
  Built:            Wed Oct 19 02:58:31 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.6.8
  GitCommit:        9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc:
  Version:          1.1.4
  GitCommit:        5fd4c4d144137e991c4acebb2146ab1483a97925
 docker-init:
  Version:          0.19.0
  GitCommit:

$ docker buildx version
github.com/docker/buildx v0.9.1 ed00243a0ce2a0aee75311b06e32d33b44729689

[steve-thousand]

I am also seeing this issue using rancher desktop

➜ ~ docker version
Client:
 Version:           20.10.21-rd
 API version:       1.41
 Go version:        go1.18.7
 Git commit:        ac29474
 Built:             Tue Nov 22 22:21:43 2022
 OS/Arch:           darwin/arm64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.20
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       03df974ae9e6c219862907efdd76ec2e77ec930b
  Built:            Wed Oct 19 02:58:31 2022
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          v1.6.8
  GitCommit:        9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc:
  Version:          1.1.4
  GitCommit:        5fd4c4d144137e991c4acebb2146ab1483a97925
 docker-init:
  Version:          0.19.0
  GitCommit:
➜ ~ docker buildx version
github.com/docker/buildx v0.9.1 ed00243a0ce2a0aee75311b06e32d33b44729689

I find I am able to get past this bug if I provide my local .docker path like so but this is not very portable

<configFile>/Users/ME/.docker/config.json</configFile>
<dockerStateDir>/Users/ME/.docker</dockerStateDir>
<platforms>
  <platform>linux/amd64</platform>
  <platform>linux/arm64</platform>
</platforms>

[andrewstein]

Further investigation done with @jflinchbaugh

The following does work:

                <buildx>
                  <dockerStateDir>~/.docker</dockerStateDir>
                  <platforms>
                    <platform>linux/amd64</platform>
                    <platform>linux/arm64</platform>
                  </platforms>
                </buildx>

After looking into it is seems that the --config argument to docker, in this case supplied via dockerStateDir must point to an actual directory with a cli-plugins/docker-buildx executable or link to an executable. If we do not give a dockerStateDir then the d-m-p gives a --config that is empty. We believe that this is the source of the bug.

[chonton]

If you can reproduce the difference in behavior between Docker's cli and Rancher's cli, then it's a Rancher problem. I am not inclined to work around this deviation. You may wish to create an issue in the appropriate Rancher repository.

[steve-thousand]

I created a separate issue when I discovered that pointing the config to my local docker dir <dockerStateDir>~/.docker</dockerStateDir> was overwriting my config.json file here https://github.com/fabric8io/docker-maven-plugin/issues/1653

I did some further investigation and found that if the code is changed to not provide any config information or config.json path information to docker then it builds successfully for me. I wonder if it could be possibly through configuration to have the plugin call docker without specifying a config and have it use docker's default local configuration.

Am I misunderstanding the purpose of the configuration here?

[chonton]

The state of the docker buildx builder instances is stored in the configuration directory. The intent of docker-maven-plugin using a non-default configuration directory is to isolate the builder instances of different projects. Additionally, without this isolation, I had hundreds of builder instances in my default configuration.