docker-compose-buildkite-plugin icon indicating copy to clipboard operation
docker-compose-buildkite-plugin copied to clipboard

No ability to retry on failure to pull images for dependencies

Open yob opened this issue 3 years ago • 0 comments

Occasionally when running a step with this plugin, it fails with an error like this. Setting the pull-retries plugin option has no effect.

:docker: Starting dependencies
$ docker-compose -f docker-compose.yml -p buildkitea1243 -f docker-compose.buildkite-132600-override.yml up -d --scale test=0 test
Creating network "buildkitea1243_default" with the default driver
Creating volume "buildkitea1243_redis-data" with default driver
Creating buildkitee371b94f5d54410598a52082c25fa46c_redis_1          ... done
Pulling redis (redis:4.0.5-alpine)...
4.0.5-alpine: Pulling from library/redis
1160f4abea84: Pull complete
a8c53d69ca3a: Pull complete
2dbffccae30e: Pull complete
8e070aa3b2f1: Pull complete
3e7d4e53fbda: Pull complete
c4dd5493c595: Download complete
ERROR: error pulling image configuration: Get https://...... net/http: TLS handshake timeout

It seems like the image for the main service is pulled here, and pull-retries can be used to add some resilience to temporary errors: https://github.com/buildkite-plugins/docker-compose-buildkite-plugin/blob/3e153720e297a3ef1e86891f69cc0ac5e416ea7d/commands/run.sh#L73

After the image for the main service has been pulled, we then attempt to start the dependent services (redis in this example) which may involve docker-compose automatically fetching this images over the network: https://github.com/buildkite-plugins/docker-compose-buildkite-plugin/blob/3e153720e297a3ef1e86891f69cc0ac5e416ea7d/commands/run.sh#L153-L158.

Is it feasible to add the retry logic in the dependency pulls as well? Maybe by reusing the pull-retries option?

yob avatar Jul 07 '20 14:07 yob