cli
cli copied to clipboard
terraform-compliance
Flagging on module use and disuse and flagging on source of said module using metadata
Determine module use and module used for a given resource, with the goal of failing on use or disuse of a module for a certain resource type in the same manner the Change:Action metadata allows to filter on "create","update","no-op", etc
module Use filtering: Current Json Parsing allows for the ingress if the change:Action: metadata, is it possible to also incorporate the following data to the resources in the stash:
The desire in this situation would be to ensure that all s3 buckets are created using not only a module, but our specific module. Since the address of the resource in the stash only points to an address that starts with module. there are limitations within the resource blocks
Given resource blocks :
module "moduletester" {
source = "[email protected]:fubarcorp/tf-module-s3-bucket?ref=v1.8.1"
bucket_name = "module-bucket-tester"
bucket_region = "us-west-2"
group = "yeah"
portfolio = "ok"
source_tag = "yolo"
}
resource "aws_s3_bucket" "nonmodulebucket" {
bucket = "nonmodule-bucket"
acl = "private"
tags = {
Name = "My bucket1"
Environment = "Dev"
}
}
what this produces The module produces a bucket and a backup bucket as well as a large amount of specific configurations, currently I am using negative testing to account for all scenarios where a bucket could vary from what the module produces and fail those outliers under a larger overarching feature to "please use our s3 bucket module".
Describe alternatives you've considered :
In the actual plan json, if metadata can be pulled into the resources to add in if they are created with the root_module versus module_calls and then reference the source, we should be able to filter in the same manner that we do for change action. See below for this block within the json
"root_module": {
"resources": [
{
"address": "aws_s3_bucket.nonmodulebucket",
"mode": "managed",
"type": "aws_s3_bucket",
"name": "nonmodulebucket",
"provider_config_key": "aws",
"expressions": {
"acl": {
"constant_value": "private"
},
"bucket": {
"constant_value": "nonmodule-bucket"
},
"tags": {
"constant_value": {
"Environment": "Dev",
"Name": "My bucket1"
}
}
},
"schema_version": 0
}
],
"module_calls": {
"moduletester": {
"source": "[email protected]:fubarcorp/tf-module-s3-bucket?ref=v1.8.1",
"expressions": {
"bucket_name": {
"constant_value": "module-bucket-tester"
},
"bucket_region": {
"constant_value": "us-west-2"
},
"group": {
"constant_value": "yeah"
},
"portfolio": {
"constant_value": "ok"
},
"source_tag": {
"constant_value": "yolo"
}
},
"module": {
"outputs": {
"backup_bucket_arn": {
"expression": {
"references": [
"aws_s3_bucket.backup"
]
}
},
"backup_bucket_id": {
"expression": {
"references": [
"aws_s3_bucket.backup"
]
}
},
"backup_bucket_region": {
"expression": {
"references": [
"aws_s3_bucket.backup"
]
}
},
"main_bucket_arn": {
"expression": {
"references": [
"aws_s3_bucket.main"
]
}
},
"main_bucket_id": {
"expression": {
"references": [
"aws_s3_bucket.main"
]
}
},
"main_bucket_region": {
"expression": {
"references": [
"aws_s3_bucket.main"
]
}
},
"main_bucket_regional_domain_name": {
"expression": {
"references": [
"aws_s3_bucket.main"
]
}
},
"main_bucket_website_domain": {
"expression": {
"references": [
"aws_s3_bucket.main"
]
}
}
},
Example Feature would be written as such:
Given I have aws_s3_bucket defined
When its actions metadata has create
And its modules metadata does not have [email protected]:fubarcorp/tf-module-s3-bucket?ref=v1.8.1
Then the scenario should fail
This is very simplified and something like
And its modules metadata does not have [email protected]:fubarcorp/tf-module-s3-bucket?ref=v1.8.1 referenced
also works.
Let me know if this has any grounds for possible addition, we see it as a very powerful filter mechanic to guide toward standard module use and specific modules to be used.
Awesome, let me know if I can help. writeup on our deployment will be available soon!
@eerkunt any luck playing around with this? I have some working features based on negative testing against a series of arguments based on our modules intended output, however it would be a much shorter feature if we were able to flag this here and could open up a bit more granularity for some of the issues folks have with modules
Apologies @mdesmarest , I had a long and tiring battle with Covid-19 for the last 4 weeks. Couldn't have a look on anything really.
Will try to regain my energy and start to take care of issues somewhere next week.
:(
@eerkunt No need to apologize! sorry to hear, glad you are coming out the other side of the worst of it! Get better. Trying to take a stab at some of the low hanging fruit issues involving BDD logic flaws and feature optimization.
Just recovered and looking into some of the issues with this, I have a feeling with the current version, this looks doable utilising @precondition('<SCENARIO NAME>') where we define module & resource that uses module in two scenarios where one of them is a precondition to other.
I will get my head around this and will try & update here.
Hmm thinking this again, I think we need to have a new type support in GIVEN steps for this : module :) Which can be used specifically for the metadata of the module-related actions. 🤔
@eerkunt
Hmm thinking this again, I think we need to have a new
typesupport inGIVENsteps for this : module :) Which can be used specifically for the metadata of the module-related actions. 🤔
Thanks for looking into this!
⬆️ This is exactly what i was thinking. Precondition sounds like the most ideal but to drill down to the actual name of the module as source we do not have that drill down ability just yet.
Also with this metadata you can filter root_module vs module_calls in terms of metadata to enforce on if a resource should not be created with a module as well as enforcing when a certain resource is created it must use a specific module.
@eerkunt Just wanted to check in to see if you were playing with this at all in terms of POC, either how you had proposed this via given or the possibility of adding this into the metadata in the stash the same way change:action was used. and we referenced it in when. just scheming up some possible features with this potential capability in mind. Thanks again for all the help, we know this is a labor of love on your part.
@eerkunt awesome! I see you were tied up in a whole bunch of issues. Please let me know if there is any further context I can provide to help out. Thanks again for taking a crack at this
@eerkunt Just wanted to check in to see if you made any progress on this or were able to disqualify this as a possible feature. I am scaling out my features and scenarios and wanted to get an idea of where things are headed. once again thanks for looking into this for us.
Hi @mdesmarest , yes it looks like this will be on the next release along with #478
@mdesmarest
I will look into this issue. Before doing so I wanted to double check:
Given I have aws_s3_bucket defined
When its module_address metadata is module.moduletester
Do these steps filter the bucket correctly? If so, is this what you were looking for or do you specifically need the source part to be checked for? (e.g. to see if it's being pulled from the correct address.)
If the latter is the case, I can add source to the metadata.
If you could share the complete plan file, I could also use it as a test and example use case.
@eerkunt Sorry for the delay, I was off for a week.
The source is the key metric to flag on as this creates a granular place to determine the actual module being used. module_address can be dictated by factors within the Terraform code itself. The source allows identification of not only a module is being used, but WHICH module specifically is being used. This is key to regulating that in this case an S3 bucket is being built with a module, AND the module is built with a specific source. The idea here is that we do not have to do a large set of negative tests to use what the module would build versus a non supported module. This also makes it much easier to update features when aspects of the module change.
This type of filtering would be used to greatly simplify the features we write to enforce specific modules and not have to account for options to configure within this module.
I can certainly supply the plan file, but would prefer to post it to a private discussion rather than the public issue.
As for use case as stated above, the natural path of compliance as code would be to graduate to module useage to allow conformity and easy update.
If you want to open a private discussion, I will submit a simple plan file containing a bucket created without module and with module to show the scenario i posted above. Thanks again!
@eerkunt let me give you a clearer usecase for understanding, without being able to match against source, I have to write features like
# Verifies an s3 bucket being newly created
Background:Check for the presence of a new aws_s3_bucket
Given I have aws_s3_bucket defined
When its actions metadata has create
#verifies an s3 bucket is being created with a module
Scenario: Ensure a module is used on all new s3 buckets
Then it must have "module." referenced. <*****************
#verifies the presence of server_side_encryption
Scenario: Ensure server_side_encryption enabled on all new s3 buckets
Then it must have server_side_encryption_configuration
#verifies encryption at rest is set to AES256
Scenario: Ensure that all new s3 buckets contain sse_algorithm and it must be set to AES256
When it has server_side_encryption_configuration
Then it must have sse_algorithm
And its value must be AES256
Scenario: Verify replication_configuration is setup on primary bucket and using the correct role
When it does not contain aws_s3_bucket
Then it must have replication_configuration
Then its role must be arn:aws:iam::REDACTED:role/S3-Cross-Account-Backup
Scenario: Verify a backup bucket is created and mapped to the primary bucket using the correct role
When it does not contain replication_configuration
Then it must have aws_s3_bucket
Then it must have replication_configuration
Then its role must be arn:aws:iam::REDACTED:role/S3-Cross-Account-Backup
Scenario: Verify versioning is setup on primary bucket and set to true
When it does not contain aws_s3_bucket
Then it must have versioning
Then its enabled must be true
Scenario: Verify versioning is setup on backup bucket and set to true
When it does not contain replication_configuration
Then it must have aws_s3_bucket
Then it must have versioning
Then its enabled must be true
This leaves a hole interms of our desired design pattern, since if we alter our module, we may fail some of these options. Since I have to test against the desired output of a module, this only works if a module is rigid, not if a module may have the absence of or inclusion of some optional parameters. I can only infer with a general certainty that we are using our module, also creates a potentially large set of failures to read through.
Module address while useful, many times is not a static value, since in the case of multiple s3 buckets, the module name will take on the name within the resource blocks so we are able to deploy more than one bucket that doesn't share a resource name. This does not allow me to specifically filter on module address
Background:Check for the presence of a new aws_s3_bucket Given I have aws_s3_bucket defined When its actions metadata has create Then it must have "module." referenced And its source meta data is "module source url address"
this allows a much cleaner feature that does not need any of the other checks, and when we move up a module version or change the module itself, we do not need to repattern the actual feature failures. If we alter our encryption or replication parameters the entire feature needs to be rewritten to pattern that. Depending on the resource you may need to write more tests.
Using Source lets us leverage DEDUCTIVE reasoning , the current pattern is INDUCTIVE as in " If it walks like a duck, swims like a duck, sounds like a duck, looks like a duck" its probably a duck, but we cant be totally sure. Source is key and DEDUCTIVE
I run Terraform-Compliance on atlantis, and run the output through a bash script that on output feeds:
` Feature: s3 buckets must be created using the 2U module. Details: https://github.com/REDACTED/tfcompliance/all/s3/s3module.feature Fixes: Resources may fail one or several checks. Any failures indicate disuse of the 2u s3 module, or the use of an unapproved s3 module Module Alert: For all new s3 buckets, please use: https://github.com/REDACTED/tf-module-s3-bucket
` We can point users to the feature path for details, provide a list of intended fixes, and in the case of module use, point them to our module. The features get exponentially more difficult to write when any of the resource parameters are optional or there are several acceptable choices.
where should you fail any of the tests within this long set of negative tests, this indicated that while you may be using a module, you probably aren't using ours.
There is no way to actually test that users are using the specific module we wish them to, to allow uniform updates when we make changes to our, in this case s3 bucket baseline.
Not only does this create a cleaner looking feature, it creates a 100% check. You can make the module address nearly anything you wish based on how the module works, and as I stated the module address may be different if you are using it more than once, but the module source allows for the conditional to enforce on 1 or several acceptable source paths, and these are finite and absolute versus inferential
Please let me know if you need more info.
Hi @mdesmarest
I added source to metadata on #532. Could you give it a try?
@mdesmarest is it all ok for this issue ? I would like to close it if its resolved.