terramate

terramate copied to clipboard

Is your feature request related to a problem? Please describe. to manage dependencies efficiently, it may be useful to introduce a "stack move" command.

Describe the solution you'd like tm stack move <path to source stack| uniquely identifiable tags> <path to destination> as part of the move:

1. validate stack's dependencies
2. update the 'before/after' in all other stacks according to this stack's new location in an interactive session.
3. the move, as well as config updates, should be an atomic operation.

The "move" can also serve as a "rename" with the same functionality.

Describe alternatives you've considered potential alternative

This makes total sense..

@i4ki i noticed using tags is not fully covered in docs.

Could we add examples and documentation how to use tags instead of path?

This can then be used to work around the issue until move is available.

@i4ki (?) pointed out that tags can not take a variable (for now). And, dir specification made a lot of sense to me. I don't think anyone will object to additional documentation :)

@az-z @mariux indeed the documentation is not good. Here it just briefly says that Tag filters can be used in after and before: https://github.com/mineiros-io/terramate/blob/main/docs/stack.md#stackafter-setstringoptional

The Tag Filter docs don't mention how it should be used in the stack.after, etc.

I'm gonna improve the docs ASAP.

Meanwhile, have a look in the example below:

stack {
    after = [
        "tag:network,iam",
    ]
}

The config above declares a stack that must run after the stacks tagged with network OR iam.

The syntax after the tag: is defined by the Tag Filter docs I linked above.

We improved the documentation of using tag: in the ordering: https://terramate.io/docs/cli/stacks/#stack-after-set-string-optional

Hi @az-z

did using tags solve your initial issue? If so, would it be okay to close this request?

Closing now, please re-open when this did not solve your poblem.

@mariux , apologies for not updating the ticket and not replying to your messages. Exactly yesterday i was creating a new environment and recalled your message here. Here is a quick test i did.

some background. the repo is structured in the following way: client/project/env/infra-component. for this project the stacks are : SOPRO/D9/PROD/cicd, SOPRO/D9/PROD/comp SOPRO/D9/PROD/elb

each environment has "globals.tm" that defines and assigns variables for this particular environment: SOPRO/D9/PROD/globals.tm

the "PROD" is fully functional code - it was deployed yesterday with TM's help. The code does not use TM's tags. e.g:

simplified PROD/globals.tm 
# Globals for D9 PROD environment
globals {

   env_abbr = "Prod"
 ...
   launchtempl = {
       AMI = tm_upper("${global.client_abbr}_${global.product_abbr}*")
....
   }
}

each stack has `stack.tm.hcl that generates terraform.tfvars, e.g:

more ./SOPRO/PROD/comp/stack.tm.hcl 

stack {
  name        = "D9-PROD-comp"
  description = "VMs resources for D9 prod  env"
  after = [
    "/SOPRO/config/keys",
    "/SOPRO/config/Security",
    "/SOPRO/config/S3",
    "/SOPRO/D9/common",
    c
  ]
}

generate_hcl "terraform.tfvars" {

  content {
...
    launchtempl = global.launchtempl
    env_abbr = global.env_abbr
    subnet_names = global.subnet_names
...
  }
}

as you can see there are 3 types of dependencies on the stack level:

1. the general client level - " "/SOPRO/config/Security"," -> these should be executed before anything specific to project and project/env can be ran;
2. project specific - "/SOPRO/config/S3";
3. environment specific -"/SOPRO/D9/PROD/elb" - the prod stack in D9 project should be executed before d9/prod/elb can run.

I hope, that with this information, the following errors should be less cryptic:

terramate experimental clone PROD PreProd
Cloned 3 stack(s) from PROD to PreProd with success
Generating code on the new cloned stack(s)
Code generation report

Successes:

- /SOPRO/D9/PreProd/cicd
	[~] backend.tf
	[~] terraform.tfvars

Failures:

- /SOPRO/D9/PreProd/comp
	error: /SOPRO/D9/PreProd/comp/stack.tm.hcl:16,19-37: evaluating content: generate_hcl "terraform.tfvars": partial evaluation failed: /SOPRO/D9/PreProd/comp/stack.tm.hcl:16,25-37: eval expression: This object does not have an attribute named "launchtempl".

- /SOPRO/D9/PreProd/elb
	error: /SOPRO/D9/PreProd/elb/stack.tm.hcl:28,24-47: evaluating content: generate_hcl "terraform.tfvars": partial evaluation failed: /SOPRO/D9/PreProd/elb/stack.tm.hcl:28,30-47: eval expression: This object does not have an attribute named "elb_subnet_names".

Hint: '+', '~' and '-' mean the file was created, changed and deleted, respectively.

I truncated the full paths in the message above.

Issues:

1. SOPRO/D9/PROD/globals.tm was not cloned ( there is not SOPRO/D9/PREPROD/globals.tm )
2. (I think because of that) the code generation errors are present
3. the "after" and "before" references were not updated to "preprod".

Next, i'll add tags and repeat the clone operation. I'll report the results shortly.

P.S. terramate --version 0.6.3

the changes i made: in d9/prod/elb stack.tm.hcl:

stack {
...
tags = [ "sopro", "d9", "prod", "elb"]
...
 before = [ 
"tags:sopro:d9:cicd", 
#"/Clients/SOPRO/D9/PROD/cicd",
...
]
...
}

in d9/prod/cicd stack.tm:

stack {
...
tags = [ "sopro", "prod", "d9", "cicd" ]
...
}

after `clone` into PREPROD, none of the tags got updated. 
There were no stack id generated.

I do not see much value in using tags in this particular project.  If I have to write out all 4 values ( client, project, env and stack) then it is same effort as to write the directory structure. In addition, using `path`, i know exactly if I miss anything.

I see the value of `clone` vs `cp -pr` in adding a unique stack id. This way, if I forgot to update a new stack the TM will notify me during deployment.

Hope this helps.

Thanks for reporting this @az-z, I just reopened the issue. Let me look at this with the team tomorrow and get back to you!