Is the cache cleanup lifecycle working?

[kayman-mk]

trafficstars

I think there are some issues with cleaning up the S3 cache.

S3 Lifecycle Prefix

After installing a fresh Gitlab Runner and S3 cache I noticed that the prefix within the S3 cache is project and not runner. But this is the default setting when a new cache is created. This was also mentioned in #200 This could be

• a configuration issue of the user (but I didn't found any special configuration on my side)
• a change within the Gitlab Runner project (renamed cache directory) which we didn't notice

https://github.com/npalm/terraform-aws-gitlab-runner/blob/5b6e342467557ca3e5c6567ee01fb3b0d2ccc5de/modules/cache/variables.tf#L59-L63

S3 Lifecycle

The lifecycle rule touches files within the bucket not its content. This means that objects which are not updated (no longer in use, e.g. caches from deleted Gitlab projects) are deleted. But I think this does not happen very often as the majority of all projects in Gitlab is never deleted or archived.

What I see in my cache is that it constantly grows as old objects within the cache zip file are never deleted. This results in higher storage costs and slower pipelines as the amount of data grows over the time.

Suggestion

1. Change the default setting of the lifecycle prefix as mentioned above to activate the clean up.
2. Clean the cache files, e.g. unzip the cache, delete old files, zip and upload the cache again.

What do you think?