modernisation-platform
modernisation-platform copied to clipboard
ministryofjustice
Initial setup macie instructions
A reference to the issue / Description of it
New instructions on setting up Macie. Also includes the index to post it to the modernisation-platform instructions.
I have also included some instructions on setting this up using Terraform. This has been tested in Cooker and sets up Macie and includes a sample job. The account should be included in the MemberInfrastructureAccess role so this can be added.
How does this PR fix the problem?
Information
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
I don't think we should be doing this via the console, it looks like there are all the resources needed to do it via code, did you come across any issues which would prevent us doing it via code?
I did have a look for terraform code to do the implementation and creating the jobs but it was all very limited. The code to create an account was straightforward but linking to other accounts or picking up the correct buckets etc. seemed a little light.
I don't think we should be doing this via the console, it looks like there are all the resources needed to do it via code, did you come across any issues which would prevent us doing it via code?
I did have a look for terraform code to do the implementation and creating the jobs but it was all very limited. The code to create an account was straightforward but linking to other accounts or picking up the correct buckets etc. seemed a little light.
In your guidance I can see you enable Macie, which looks to be possible with this - https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/macie2_account
And then create a job which looks to be possible with this: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/macie2_classification_job
So it seems possible to do what you are advising in the console as code, what were the issues when you tried this?
I haven't actually tried the terraform code, I just looked at it an it looked very limited. I did state that it should be done in terraform and included the examples to do this. I could give it a go but it would mean opening up Macie elsewhere (30 day trial). Ewa suggested setting it up in one place to link all of the buckets we want to examine in one place but I think this would also involved setting up Macie in each account you want to use. Again, many account and who would then look at the Macie results? Not us. If you want I can set up the terraform code in modernisation-platform repo to test it against (somewhere) but I need a location and confirmation you want me to do this. Oh, and I can then post a link in the documentation to some example (maybe use example?) code.
I haven't actually tried the terraform code, I just looked at it an it looked very limited. I did state that it should be done in terraform and included the examples to do this. I could give it a go but it would mean opening up Macie elsewhere (30 day trial). Ewa suggested setting it up in one place to link all of the buckets we want to examine in one place but I think this would also involved setting up Macie in each account you want to use. Again, many account and who would then look at the Macie results? Not us. If you want I can set up the terraform code in modernisation-platform repo to test it against (somewhere) but I need a location and confirmation you want me to do this. Oh, and I can then post a link in the documentation to some example (maybe use example?) code.
Hi Steve, I wasn't aware you can enable it through code, so if that works, it then does not matter much if we control it from the OU account (like modernisation-platform) or from the member account itself. I would suggest try it with terraform in one of our playground accounts and if that works, then we can roll it out to our users through the member-bootstrap perhaps (or maybe @davidkelliott is aware of a better place for that). If cooker account is the one you used last time, try it in sprinkler or example.
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
@ewastempel and @davidkelliott I have amended the instructions to list some terraform code (that works if the prerequisite is completed) and posted this. I might need to add something at the top that references this perhaps.
Extra line with a comment to reference the terraform added
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
I can still see instructions on how to do manually, unless this is absolutely essential, can this be removed please? We are a infra as code first platform, thanks
Will do. I will keep them in place at the bottom unless you have any objections. We may need to look up where the role is set to make sure that is updated but I have not included that in this documentation.
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
I don't think we should be doing this via the console, it looks like there are all the resources needed to do it via code, did you come across any issues which would prevent us doing it via code?
Apart from the issue with making sure that the ID has access to the macie2:* I saw no other issues. This, I think, is mentioned in the pre-requisites.
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
I've added a few changes, I think this needs to be an introduction to what Macie is and does, and a link to code on how to add it to their account.
Where are the changes? Do I need to do a pull to drag them in or have you amended my release?
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
I've added a few changes, I think this needs to be an introduction to what Macie is and does, and a link to code on how to add it to their account.
I can't see these changes (a release has overwritten them I think). Can you let me know what they were and I will manually put them in or re-enter them into the PR and I will add them
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Not to come across as too critical but I don't think this documentation adds a lot of value. We don't go into detail like this on everything we do in our example account. I can appreciate that you want to produce meaningful outputs like this, but a lot of this could be summarised as such:
1. Customers can set up Macie on a per-account basis.
2. We should be aware that this can incur costs.
3. You can see an example of functioning Macie code in example [here](https://a-link-to-the-macie-code-in-example).
4. AWS has documentation [here](https://some-kind-of-link-to-macie-documentation).
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
To @dms1981 Originally I thought you could only do this via the console. After investigating I found it could be done, more easily, through terraform. This changed the scope a fair bit. I have kept a copy of the detail I originally wrote in case I need it again but I understand your points
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders:
Trivy Scan Success
Show Output
```hclTrivy will check the following folders:
</details> #### `Checkov Scan` Success
<details><summary>Show Output</summary>
```hcl
*****************************
Checkov will check the following folders:
CTFLint Scan Success
Show Output
*****************************
Setting default tflint config...
Running tflint --init...
Installing "terraform" plugin...
Installed "terraform" (source: github.com/terraform-linters/tflint-ruleset-terraform, version: 0.5.0)
tflint will check the following folders:
Trivy Scan Success
Show Output
*****************************
Trivy will check the following folders: