4.22.0 - docker-machine install fails and config file has __REPLACED_BY_USER_DATA__

[cmavromichalis]

Upgrading to 4.22.0 our runners would not register with GitLab. The config.toml file had

token = "__REPLACED_BY_USER_DATA__"

Looking in journalctl I see these errors:

Feb 22 20:54:49 user-data[2844]: ++ echo https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.10/docker-machine-Linux-aarch64
Feb 22 20:54:49 user-data[2844]: + [[ https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.10/docker-machine-Linux-aarch64 == '' ]]
Feb 22 20:54:49 user-data[2844]: + curl --fail --retry 6 -L https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.10/docker-machine-Linux-aarch64
Feb 22 20:54:49 user-data[2844]:   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Feb 22 20:54:49 user-data[2844]:                                  Dload  Upload   Total   Spent    Left  Speed
...
Feb 22 20:54:51 user-data[2844]: + chmod +x /tmp/docker-machine
Feb 22 20:54:51 gitlab-runner[7209]: ERROR: Machine creation failed                      error=exec: "docker-machine": executable file not found in $PATH name=runner-replac-
...
Feb 22 20:54:51 user-data[2844]: + docker-machine --version
Feb 22 20:54:51 user-data[2844]: /var/lib/cloud/instance/scripts/part-001: line 128: /usr/local/bin/docker-machine: cannot execute binary file

If I login as root and try to run docker-machine --version I get the same error. If I download the file I get the error as well

$ curl https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.10/docker-machine-Linux-aarch64 > docker-machine
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 41.5M  100 41.5M    0     0  15.4M      0  0:00:02  0:00:02 --:--:-- 15.4M
$ chmod +x docker-machine
$ docker-machine --version
-bash: /usr/local/bin/docker-machine: cannot execute binary file

We reverted to version 4.20 and the problem vanished.

I think this change

[npalm]

I have just deployed several times the default example to chack PR's. Docker machine was running withou an isse. Also checked the version.

[root@ip-10-0-1-211 bin]# docker-machine --version
docker-machine version 0.16.2-gitlab.1, build bd580896

In version 4.21 docker machine is switch to the GitLab version. Via the variables you can still enforce to use the default one.. If you set the variable docker_machine_version the gitlab version should be ignored.

Hope this will help

[bishtawi]

I also ran into this issue and I think its the same underlying issue as what is being reported in https://github.com/npalm/terraform-aws-gitlab-runner/issues/299

Basically, the default value for variable docker_machine_download_url points to a aarch64 binary (ARM 64) but the default docker_machine_instance_type is m5.large which is x86_64 architecture.

Once I set docker_machine_download_url to https://gitlab-docker-machine-downloads.s3.amazonaws.com/v0.16.2-gitlab.10/docker-machine-Linux-x86_64, I was able to get the Gitlab Runner instance functioning correctly.

[npalm]

Agree, is a bug, for the short run we should update the default to the x64 one.

[mateuszkwiatkowski]

I also experienced this issue. Upgrade to 4.24.1 fixed it.

[npalm]

Thx for testing. In case you have time feel free to submit a PR. Otherwise I will have a look in the weekend

[cmavromichalis]

Looks like this issue came up again with our forked version of 4.25.0 and GitLab 14.3.0

Will try to update with latest and see if that fixes it.

[kayman-mk]

4.35 is working without problems here.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days.

[kayman-mk]

The token problem was fixed with #574