feat!: Configure the S3 cache region

[kayman-mk]

trafficstars

Description

It might be useful to create the S3 cache bucket in a different region. This PR introduces the aws.cache_bucket provider to be able to create the S3 bucket using a different provider.

Closes #367

Migrations required

You have to add a aws.cache_bucket provider to the main module as well as to the cache module, e.g.

module "runner2" {
  source = "../../"

  environment = "${var.environment}-2"

  ...

  # creates the cache in the same region as the Runner and Executor
  providers = {
    aws = aws
    aws.cache_bucket = aws
  }
}

module "cache" {
  source = "../../modules/cache"

  environment = var.environment

  providers = {
    aws              = aws
    aws.cache_bucket = aws
  }
}

Input variable aws_region is now obsolete and calculated internally.

Changing The Region

This is not easy as you can't change the region of the aws.cache_bucket and reapply everything. Usually you do the following steps:

• remove the cache module from your Terraform code and apply the changes
• change the region of the aws.cache_bucket_provider
• add the cache module again and apply it.

If you remove the cache, do not forget to set the

  cache_bucket = {
    create = false
    policy = ""
    bucket = ""
  }

Verification

• [x] Create the cache in the same region as the runner. Check upload and download.
• [x] Create the cache in another region as the runner. Check upload and download.

[kayman-mk]

@npalm @welderpb Due to the fact that the Terraform language changed in 0.15.0 we can't support the old versions 0.13 and 0.14 as we have to specify configuration_aliases which is not understood by them.

I think we have three options now:

1. drop support for Terraform 0.13 and 0.14
2. do not support a different S3 cache location (other provider)
3. maintain 2 versions of the module

[npalm]

@kayman-mk dropping support for versions pre 0.15 is certainly OK. Terraform 1.x is strongly advised. If we can keep 0.15 working for some time it would be great. So feel free to drop the support for TF 0.13 and 0.14

[kayman-mk]

Totally forgot to mention my favorite: Get rid of 0.13 and 0.14. Sounds good @npalm

[kayman-mk]

@npalm Strange. As soon as I remove 0.13 and 0.14 from ci.yml no jobs are run. I have no clue, why.

[kayman-mk]

Uhhh, now it's working.

[kayman-mk]

rebased

[kayman-mk]

@npalm Fixed that now. Shall we proceed here? I'd like to do the checks again. Just to make sure that everything is fine.

[npalm]

@kayman-mk sorry no time yet to check this PR.

[kayman-mk]

@npalm Module tested.

[kayman-mk]

Branch updated

[github-actions[bot]]

This PR is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 15 days.

[kayman-mk]

@npalm Shall we close it? Was a nice programming task using providers but the supporting issue #367 still has no business reason mentioned. And I do not see any valid reason why I should create the cache in another regions than my runners. Am I overlooking something?

[npalm]

Agree I also don't see any reason to run a cache in another region. So please go ahead with closing the issue.