AWS Docker Autoscaler: error with Docker Autoscaler Executor and EC2 Instance Connect

[vpotap]

trafficstars

I am encountering an occasional issue while using the Docker Autoscaler executor. Sometimes, the following errors are logged, indicating problems with EC2 Instance Connect and the state of the EC2 instance:

Preparing the "docker-autoscaler" executor
ERROR: Failed to remove network for build
ERROR: Preparation failed: error during connect: Get "http://internel.tunnel.invalid/v1.24/info": preparing environment: getting instance connect info: refreshing connect info: rpc error: code = Unknown desc = sending ssh key: operation error EC2 Instance Connect: SendSSHPublicKey, https response error StatusCode: 400, RequestID: <here-id>, EC2InstanceStateInvalidException: Unable to connect to the instance. EC2 Instance Connect is not supported on a terminated instance. (docker.go:779:0s)
Will be retried in 3s ...
ERROR: Failed to remove network for build
ERROR: Preparation failed: error during connect: Get "http://internel.tunnel.invalid/v1.24/info": preparing environment: getting instance connect info: refreshing connect info: rpc error: code = Unknown desc = sending ssh key: operation error EC2 Instance Connect: SendSSHPublicKey, https response error StatusCode: 400, RequestID: <here-id>, EC2InstanceStateInvalidException: Unable to connect to the instance. EC2 Instance Connect is not supported on a terminated instance. (docker.go:779:0s)

The issue occurs occasionally, and re-running the job typically resolves it.