Using hclwrite to edit attributes in blocks

[nishkrishnan]

I am currently trying to change a lot of hcl configs at once. Each config looks something like follows:

module "roles-workspace" {
  source = "../../../../../../modules/workspace"
  org_name = var.org_config.name
  vcs_oauth_token_id = var.org_config.vcs_oauth_token_id
  read_only_team_ids = var.org_config.read_only_team_ids
  write_team_ids = var.org_config.write_team_ids

  workspace_cfg_map = {
    service_name = "roles"
    repo_path = "roles"
    repo_branch = "master"
    terraform_version = "0.12.2"
    workspace_suffix = "roles"
    tf_config_root = "ops/terraform/roles"
    auto_apply = false
  }
}

I'm trying to basically mark auto_apply = true and rewrite the config to the same file. HCLWrite is supposedly the best way to achieve this since it preserves any expressions and there's no data loss. However, I can't figure out how to actually get the attribute values within a block and modify it. This is the code I've been playing around with

toBeWritten, _ := hclwrite.ParseConfig(
bytes, path, hcl.InitialPos,
)

for _, block := range toBeWritten.Body().Blocks() {
  attr := block.Body().Attributes()["workspace_cfg_map"]
  // not sure what to do at this point? How do I get the expression contents? Is there some traversal 
  that needs to happen here?
  traversals := attr.Expr().Variables()

}

I've been referencing this issue: https://discuss.hashicorp.com/t/parse-hcl-treating-variables-or-functions-as-raw-strings-hashicorp-hcl/5859

Is there any way to achieve what I'm trying to currently with hclwrite? Or do I have to accept the data loss and use gohcl

[zx8]

I just came across this issue from Google and am trying to do something very similar, and got stuck in the same way – that is, I can't figure out how to get the value of the attribute (whether it's a string, boolean, integer, etc.).

@apparentlymart Any pointers here? I've referenced the following projects with no luck:

• https://github.com/hashicorp/terraform-config-inspect
• https://github.com/apparentlymart/terraform-clean-syntax
• https://github.com/tmccombs/hcl2json (this comes closest, but casts the body to *hclsyntax.Body in order to handle it)

Is there anyway to get the values using hclwrite exclusively?

---

Edit: Found https://github.com/minamijoyo/hcledit/blob/9e07b3815ac340767a6472f850fdbe0d74c7fcf4/editor/attribute_get.go#L204-L233 which seems to work. Hopefully we can do this natively in future! 😄

[apparentlymart]

There is currently no API in hclwrite for setting only the auto_apply in there. Currently the only APIs available are to replace the entire value of an argument, which in this case would be to replace the whole object assigned to workspace_cfg_map.

Surgical edits to the inside of expressions is something that would require a lot more AST types to represent (in this case) the object constructor, the individual attribute definitions inside it, and then the value expressions of those. I don't expect to have time to design and implement such a thing in the near future, unfortunately.

For a one-shot update tool like the opening comment here seems to be describing (sorry if I misunderstood), I'd honestly probably just do it using simple string replacement and assume that all of the input files are written by reasonable humans who wouldn't be using any weird non-idiomatic syntax. While it would be nice to be able to address use-cases like that with hclwrite, that's a lot of work to implement for just a one-shot tool that would be run once and then never used again. I'd like to make it work one day, but it's difficult to prioritize that over other work right now.

[andrej-gomozov]

Having same issue and struggling to read attribute values using hclwrite and apparently it's not a current functionality at all. Would be very useful if this actually part of hclwrite. My current use case is to merge values from one Terraform module into another and not really want to do this via JSON due to readability.

UPDATE: eventually Expr().BuildTokens(nil) on Attribute value did the trick in my case and returned the actual value.

[ddouglas]

Having same issue and struggling to read attribute values using hclwrite and apparently it's not a current functionality at all. Would be very useful if this actually part of hclwrite. My current use case is to merge values from one Terraform module into another and not really want to do this via JSON due to readability.

UPDATE: eventually Expr().BuildTokens(nil) on Attribute value did the trick in my case and returned the actual value.

I have a need identical to what the OP has. An attribute with an object type and the need to preserve all of the values in the object whilst updating a value in the object. This is what I've gotten accomplished so far.

Example Terraform

provider "aws" {
  region = "us-east-1"

  default_tags {
    tags = {
      "foo" = "bar",
      "t"   = var.alpha
    }
  }
}


variable "alpha" {
  type    = string
  default = "bravo"
}

main.go

fi, _ := os.Open("./test.tf")
	fd, _ := io.ReadAll(fi)
	f, _ := hclwrite.ParseConfig(fd, "", hcl.InitialPos)

	var provider *hclwrite.Block
	for _, block := range f.Body().Blocks() {
		if block.Type() != "provider" {
			continue
		}

		if len(block.Labels()) != 1 || block.Labels()[0] != "aws" {
			continue
		}

		provider = block
		break
	}

	if provider == nil {
		logger.Fatal("failed to find provider aws")
	}

	var defaultTags *hclwrite.Block
	for _, block := range provider.Body().Blocks() {
		if block.Type() != "default_tags" {
			continue
		}

		defaultTags = block
	}

	if defaultTags == nil {
		logger.Fatal("provider does not have a default tags block")
	}

	tagsAttribute := defaultTags.Body().GetAttribute("tags")
	if tagsAttribute == nil {
		logger.Fatal("default_tags does not have the expected tags attribute")
	}

	tagsTokens := tagsAttribute.Expr().BuildTokens(nil)

Here we open the file, parse it and hunt for the default_tags block inside of the aws provider. Thanks to @andrej-gomozov, I realized that I could get a string representation of the object.

From there:

	tagsParsedExpression, diag := hclsyntax.ParseExpression(tagsTokens.Bytes(), "test.tf", hcl.InitialPos)
	if diag.HasErrors() {
		logger.Warn("hclsyntax.ParseExpression", diag.Error())
	}

	tagsValue, diag := tagsParsedExpression.Value(nil)
	if diag.HasErrors() {
		logger.Warn("tagsParsedExpression.Value", diag.Error())
	}

	tagsMap := tagsValue.AsValueMap()
	tagsMap["AppID"] = cty.StringVal("some-random-stuff") // This is what I'm attempting to accomplish. Victory!!!!

	ctyMap := cty.MapVal(tagsMap)

At this point i've mutated the map and injected the value I need. Now I need to get back some tokens so that I can overwrite the tags attributes inside the default_tags block.

	newTagTokens := hclwrite.TokensForValue(ctyMap)

This panic's. Why? Because I'm only generating code but this is expecting the value for the var.alpha variable that is in the provider block.

panic: cannot produce tokens for unknown value

goroutine 1 [running]:
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x1029ba980, 0x140000a6013}}, {0x10298eba0, 0x102af8710}}, {0x140000dc280, 0x10, 0x10})
        /Users/ddouglas/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclwrite/generate.go:189 +0x1234
github.com/hashicorp/hcl/v2/hclwrite.appendTokensForValue({{{0x1029baa28, 0x14000096b50}}, {0x102990820, 0x14000099b30}}, {0x0, 0x0, 0x0})
        /Users/ddouglas/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclwrite/generate.go:287 +0xd9c
github.com/hashicorp/hcl/v2/hclwrite.TokensForValue({{{0x1029baa28?, 0x14000096b50?}}, {0x102990820?, 0x14000099b30?}})
        /Users/ddouglas/go/pkg/mod/github.com/hashicorp/hcl/[email protected]/hclwrite/generate.go:27 +0x38
main.main()
        /Users/ddouglas/temp/manipulate-terraform/main.go:82 +0x738
exit status 2

I use the spew.Dump lib to inspect what is going on

(cty.Value) {
 ty: (cty.Type) {
  typeImpl: (cty.typeMap) {
   typeImplSigil: (cty.typeImplSigil) {
   },
   ElementTypeT: (cty.Type) {
    typeImpl: (cty.primitiveType) {
     typeImplSigil: (cty.typeImplSigil) {
     },
     Kind: (cty.primitiveTypeKind) 83
    }
   }
  }
 },
 v: (map[string]interface {}) (len=3) {
  (string) (len=3) "foo": (string) (len=3) "bar",
  (string) (len=1) "t": (*cty.unknownType)(0x100b80da0)({ <--- There is the sucker. The unknown variable
   refinement: (cty.unknownValRefinement) <nil> /// 
  }),
  (string) (len=9) "AppID": (string) (len=16) "some-random-stuff"
 }
}

I've confirmed that if I remove the variable from the default_tags block this works beautifully. I'd be okay with this implementation, but I know for a fact that we have implementations of default tags that rely on variables to be passed in so this wouldn't pass QA.

Just providing my findings. Could we write a function that breaks down cty some how and turns it back into tokens and then uses the pure raw functions to set the values? Yes. Am I going to? Probably not. We've settled on posting a disclaimer on our site that'll let our end users know we're going to blow away the default tags and replace them with our own rendered object.