terraform icon indicating copy to clipboard operation
terraform copied to clipboard

Override module resource from outside module

Open nitrocode opened this issue 4 years ago • 21 comments

Terraform Version

Terraform v1.3.2
on darwin_amd64

But the version doesn't matter here.

Use Cases

Instead of having to wait for a module owner to update a resource in their tf module and instead of having to fork the module itself to make those changes, it would be nice to override the module resource from outside the module so you have full access to all of the resource-to-override's arguments and still use most of the feature set that comes from using the module itself.

Overriding a module with a local patch is similar to how you can use kustomize to use an upstream helm chart and apply a patch. A "kustomize for terraform" but built-in so we don't have to build the feature into yet another terraform wrapper like terragrunt, atmos, terramate, etc.

Attempted Solutions

  1. terraform init
  2. Use patch to apply a local patch to .terraform dir
  3. terraform plan or terraform apply

Proposal

Example hcl but could be different. This would be a patch file that could be applied to the module when it is initialized.

module "example" {
  source  = "some/terraform-module/aws"
  version = "1.0.0"

  patches = [
    "${path.module}/example.1.0.0.disable-sg.patch",
  ]

  input_one   = "input"
  input_two   = "input"
  input_three = "input"
  input_four  = "input"

  # ...
}

Then terraform init could download the module to the terraform data dir (.terraform) and apply the patch to the downloaded module.

References

  • https://dev.to/camptocamp-ops/use-kustomize-to-post-render-helm-charts-in-argocd-2ml6
  • https://austindewey.com/2020/07/27/patch-any-helm-chart-template-using-a-kustomize-post-renderer/

Workaround

Using the patch command

# get into dir
cd components/terraform/ecr
terraform init
# backup file to patch
cp .terraform/modules/xyz/main.tf{,.orig}
# makes changes to file
vim .terraform/modules/xyz/main.tf
# create patch
diff -Naur .terraform/modules/xyz/main.tf.orig .terraform/modules/xyz/main.tf > 0-patch-terraform-dir-module-xyz-main.tf.patch
# revert
rm -rf .teraform/
# restart with init and run patch
terraform init
patch -p0 < 0-patch-terraform-dir-module-xyz-main.tf.patch
terraform plan

nitrocode avatar Jan 16 '21 13:01 nitrocode

I second this request. What is the point of using 3rd provider modules if you can not override them? As the world begins to modernize their IaC stacks, using 3rd party modules to create a more common stack templated approach, this override feature becomes critical.

elvis2 avatar Jun 11 '21 16:06 elvis2

Any updates on this? Seems like a neat feature to have.

asaranglr avatar Aug 17 '21 17:08 asaranglr

Any updates on this issue? This feature would be beneficial for using established modules that are lacking an additional modification.

danielebra avatar Sep 06 '21 06:09 danielebra

It's tempting to override resource attributes from outside the module, but it will brake "encapsulate configuration" promise. Very often modules are created to force user to create infrastructure by the rules defined by the module. Enabling overriding resource definition from outside is a bad thing in this case.

For now you can use override logic and yes, you have to fork but maintaining is much easier.

pkrolikowski avatar Feb 03 '22 20:02 pkrolikowski

Is there any updates on this? can you provide a mechanism similar to the outputs, maybe an export keyword, so you allow the caller to override resources in the module.

waltervargas avatar Jun 10 '22 10:06 waltervargas

Are there any updates about this ?

Z4ck404 avatar Oct 13 '22 08:10 Z4ck404

Dear Santa Claus,

I wish for the ability to override module resources from outside the modules too.

Thanks, Santa!

simoncpu avatar Oct 23 '22 03:10 simoncpu

There is one way to do it now

  1. terraform init
  2. Use patch to apply a local patch to .terraform dir
  3. terraform plan or terraform apply

nitrocode avatar Oct 23 '22 05:10 nitrocode

Overriding a module with a local patch is similar to how you can use kustomize to use an upstream helm chart and apply a patch.

We're basically asking for a kustomize for terraform but built-in so we don't have to build the feature into yet another terraform wrapper like terragrunt, atmos, terramate, etc.

nitrocode avatar Oct 23 '22 05:10 nitrocode

This sounds interesting. I wish this would work on Terraform Cloud though...

simoncpu avatar Oct 23 '22 06:10 simoncpu

My use case is that I reorganized my code into two modules. Both modules call cloudflare_zone on the same zone. I'm looking for a way to let the main code override the resources on both modules so that Terraform would treat it as a single resource.

simoncpu avatar Oct 23 '22 06:10 simoncpu

Yes, that would be great if terraform provides a feature like kustomize patch.

benamazing avatar Nov 22 '22 03:11 benamazing

This would still be great.

kderck avatar Feb 13 '23 20:02 kderck

This feature is required, it will help to customize the module resources on which you don't have control.

Saurabhbatham avatar Feb 18 '23 04:02 Saurabhbatham

Really hoping this gets implemented.

jdavis-politico avatar Mar 30 '23 15:03 jdavis-politico

There are many times where I've wished to reach for a feature like this. I recognize it can break some of the beauty of a well-crafted module encapsulation/abstraction, but it's not uncommon to find yourself in a situation where it just makes practical sense to be able to craft an override for a module you don't control. This is especially the case in scenarios where vendoring a fork of someone else's module actually introduces more complexity than a well crafted and commented override would.

brannondorsey avatar Apr 20 '23 03:04 brannondorsey

It's tempting to override resource attributes from outside the module, but it will brake "encapsulate configuration" promise. Very often modules are created to force user to create infrastructure by the rules defined by the module. Enabling overriding resource definition from outside is a bad thing in this case.

For now you can use override logic and yes, you have to fork but maintaining is much easier.

Back in 2021, this override feature was not fully developed. Maybe that is no longer the case now...

Saying that, I just wanted to chime and and continue beating the drum that this is a needed feature.

elvis2 avatar May 15 '23 20:05 elvis2

+1 for this feature

sanjeevpandey19 avatar Jun 26 '23 06:06 sanjeevpandey19

Thanks for your interest in this issue! This is just a reminder to please avoid "+1" comments, and to use the upvote mechanism (click or add the 👍 emoji to the original post) to indicate your support for this issue. This helps avoid notification spam for issues with high numbers of participants while enabling the maintainers to prioritize issues. Thanks again for the feedback!

crw avatar Jun 26 '23 17:06 crw

Agreed, there's a lot of use cases that this would improve! Please add this in.

liamhelmer avatar Aug 30 '23 19:08 liamhelmer