terraform-aws-mwaa icon indicating copy to clipboard operation
terraform-aws-mwaa copied to clipboard

What is the intended deployment pattern for changing requirements and plugins?

Open easontm opened this issue 2 years ago • 6 comments

I see that the lifecycle settings ignore changes to both the requirements and plugins objects

  lifecycle {
    ignore_changes = [
      plugins_s3_object_version,
      requirements_s3_object_version
    ]
  }

Is there some kind of bad/unintended behavior if these changes are not ignored? Or is this an intentional decision to encourage these these updates via the console only?

easontm avatar Mar 08 '23 06:03 easontm

@easontm The reason for ignoring these attributes is to handle updates to the plugins and requirements separately from Terraform. By ignoring these attributes, Terraform won't automatically update the aws_mwaa_environment resource when the S3 object versions change for the plugins or requirements files. This can help avoid unintended updates to the environment when these files are modified outside of the Terraform configuration.

vara-bonthu avatar Mar 15 '23 11:03 vara-bonthu

I'd like to manually specify the object version and use terraform to manage what version MWAA uses. If I understand correctly, this block makes this usecase impossible.

aignas avatar Apr 21 '23 01:04 aignas

I use terraform to upload new requirements file and want to specify the version on aws_mwaa_environment requirements_s3_object_version option.

Currently I need to go to the console and do this operation manually

dorsegal avatar May 09 '23 13:05 dorsegal

@easontm @aignas @dorsegal It does look majority is looking for this change. I am happy to go with based on your requirements. Would you like to send a PR for this? Thanks

vara-bonthu avatar May 10 '23 13:05 vara-bonthu

FWIW the current implementation works perfectly for my use case which is:

  • i have 3 deployment environments
  • all terraform config testing is done in the first (development branch)
  • once everything is working i promote to the next environment where everything is deployment from scratch using github actions.

In github actions, my workflow is:

  1. during terraform apply: create s3 bucket, use aws-ia/terraform-aws-mwaa to configure mwaa and use that s3 bucket on source_bucket_arn, set requirements_s3_path to a relative path
  2. after terraform apply, i aws s3 cp the requirements.txt file from my repo to the above s3 bucket and i also set requirements_s3_object_version with aws mwaa update-environment.
  3. that last command will cause the mwaa environment to update which takes a few minutes

I was worried this wouldn't work since I'm providing the s3 object version after terraform and the tooltip for requirements_s3_path says:

(Optional) The relative path to the requirements.txt file on your Amazon S3 storage bucket. For example, requirements.txt. If a relative path is provided in the request, then requirements_s3_object_version is required.

However, it worked perfectly.

rhyek avatar Jan 10 '24 21:01 rhyek

hi @vara-bonthu we are not keeping the older versions of requirements file in s3 bucket, which causes terraform apply fail, due to cannot find the deleted object version. Can we undo ignore_changes block to avoid this scenario? Or instead of hardcoding, making dynamic to avoid it? I am happy to submit PR, if helps.

sakomws avatar Aug 16 '24 00:08 sakomws