autoscaling-ec2-gitlab-runners-fargate icon indicating copy to clipboard operation
autoscaling-ec2-gitlab-runners-fargate copied to clipboard

Remove the need for an IAM user

Open peterkh opened this issue 4 years ago • 5 comments

Had a quick read of the docs for gitlab runner and looks like it uses the instance meta data if you don't pass in a specific user for the s3 cache setting:

https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnerscaches3-section

"Note: If any of ServerAddress, AccessKey or SecretKey aren’t specified, then the S3 client will use the IAM instance profile available to the gitlab-runner instance"

This PR removes the AccessKey and SecretKey, as well as adding the same permissions to the ecs task role.

peterkh avatar Mar 03 '20 11:03 peterkh

Thanks @peterkh - Did you manage to test this? When a job runs it returns:

16: Checking cache for default...
17: No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted. 
18: Successfully extracted cache

followed by the bucket remaining empty once the build is complete.

This is the same behaviour I experienced when trying to get this to work myself, haven't been able to find the root cause yet.

woodjme avatar Mar 04 '20 09:03 woodjme

Hi,

Looks like the env var CACHE_S3_SERVER_ADDRESS might also need to be removed:

"Note: If any of ServerAddress, AccessKey or SecretKey aren’t specified, then the S3 client will use the IAM instance profile available to the gitlab-runner instance"

GMartinez-Sisti avatar Apr 01 '20 10:04 GMartinez-Sisti

Thanks @GMartinez-Sisti

For me at least this still doesn't resolve it :(

Screenshot 2020-04-01 at 14 59 39

woodjme avatar Apr 01 '20 13:04 woodjme

The entire note suggests that the "spawner" is the one that uploads the cache, this might mean that it might be the one that needs access to that profile, instead of the created VM.

Note: If any of ServerAddress, AccessKey or SecretKey aren't specified, then the S3 client will use the IAM instance profile available to the gitlab-runner instance. In an autoscale configuration, this is NOT the machine created on demand that jobs are executed on.

GMartinez-Sisti avatar Apr 01 '20 16:04 GMartinez-Sisti

FWIW, this change as is, works perfectly for me. Edit: (applied to current master branch)

alexjfisher avatar Mar 18 '21 15:03 alexjfisher