How does `hclwrite` remove `objectelem` inside a `collectionValue` of an attribute

[magodo]

I have a piece of code using hclwrite to modify some Terraform code, in that it will conditionally remove attributes/blocks. The question raises when I encounter, e.g. the following HCL (regarded as a TF attribute):

foo = {
  bar = {
  }
  baz = "xxx"
}

(The above code is not prevalent in SDKv2 based providers, but will prevail for FW based ones as attributes are preferred to blocks)

I don't know how could I, e.g., remove the bar inside the foo attribute's expression (i.e. the collectionValue).

I've tried with something similar to below, but doesn't output what I expected:

package main

import (
	"fmt"
	"log"

	"github.com/hashicorp/hcl/v2"
	"github.com/hashicorp/hcl/v2/hclwrite"
)

func main() {
	f, diags := hclwrite.ParseConfig([]byte(`
foo = {
  bar = {
  }
  baz = "xxx"
}
`), "main.hcl", hcl.InitialPos)
	if diags.HasErrors() {
		log.Fatal(diags.Error())
	}
	foo := f.Body().Attributes()["foo"]
	expr := foo.Expr()

	tks := expr.BuildTokens(hclwrite.TokensForIdentifier("tmp"))
	ftmp, diags := hclwrite.ParseConfig(tks.Bytes(), "tmp", hcl.InitialPos)
	if diags.HasErrors() {
		log.Fatal(diags.Error())
	}

	body := ftmp.Body()
	body.Blocks()[0].Body().RemoveAttribute("bar")

	f.Body().SetAttributeRaw("foo", body.Blocks()[0].Body().BuildTokens(nil))
	fmt.Println(string(f.Bytes()))
}

[magodo]

Updated: I finally manage to do this via the following, though apparently not ideal:

package main

import (
	"fmt"
	"log"
	"slices"

	"github.com/hashicorp/hcl/v2"
	"github.com/hashicorp/hcl/v2/hclwrite"
)

func main() {
	f, diags := hclwrite.ParseConfig([]byte(`
foo = {
  bar = {
  }
  baz = "xxx"
}
`), "main.hcl", hcl.InitialPos)
	if diags.HasErrors() {
		log.Fatal(diags.Error())
	}
	foo := f.Body().Attributes()["foo"]

	tks, diags := removeExpressionAttributes(foo.Expr(), "bar")
	if diags.HasErrors() {
		log.Fatal(diags.Error())
	}
	f.Body().SetAttributeRaw("foo", tks)
	fmt.Println(string(f.Bytes()))
}

func removeExpressionAttributes(expr *hclwrite.Expression, attributes ...string) (hclwrite.Tokens, hcl.Diagnostics) {
	tks := expr.BuildTokens(hclwrite.TokensForIdentifier("tmp"))
	ftmp, diags := hclwrite.ParseConfig(tks.Bytes(), "tmp", hcl.InitialPos)
	if diags.HasErrors() {
		return nil, diags
	}

	body := ftmp.Body().Blocks()[0].Body()

	bodyAttributes := body.Attributes()
	var objectAttrTokens []hclwrite.ObjectAttrTokens
	for attrName, attr := range bodyAttributes {
		if slices.Contains(attributes, attrName) {
			continue
		}
		objectAttrTokens = append(objectAttrTokens, hclwrite.ObjectAttrTokens{
			Name:  hclwrite.TokensForIdentifier(attrName),
			Value: attr.Expr().BuildTokens(nil),
		})
	}
	return hclwrite.TokensForObject(objectAttrTokens), nil
}

Any idea about how to do this idiomatically would be appreciated!

[apparentlymart]

Hi @magodo,

Unfortunately the hclwrite.Expression type is largely just a placeholder today, since the functionality of hclwrite was primarily motivated by what Terraform needed at the time and then it was never important enough for me to be able to spend time improving it further. :confounded:

Directly fiddling with tokens is, unfortunately, probably the only available answer right now. If I recall correctly, the rudimentary expression-reformatting rules in terraform fmt work in that way, by just adding and removing raw tokens rather than actually analyzing the nested expressions.

---

The following is some assorted context I'm leaving here in case it's useful to someone who might want to try to build out a more substantial hclwrite.Expression design:

Right now a hclwrite.Expression has a very basic syntax tree inside of it: it contains a mixture of "unstructured tokens" and hclwrite.Traversal nodes.

For example, consider an expression like 1 + foo + 2. The hclwrite.Expression syntax tree for that would be something like this:

• Unstructured tokens: TokenNumberLit, TokenPlus.
• A nested hclwrite.Traversal node containing a hclwrite.TraverseName over the TokenIdent token representing foo.
• Unstructured tokens: TokenPlus, TokenNumberLit

The hclwrite package is intentionally designed so that each node "owns" a syntax tree that contains a mixture of unstructured tokens and other nodes, but the original idea behind "unstructured tokens" was to represent semantically-meaningless tokens like newlines and comments. The current incomplete hcl.Expression implementation is therefore in a sense "cheating" by treating everything except traversals as semantically meaningless tokens.

The way I'd expected this to evolve in future was that the expression parser would type-assert its nativeExpr argument to find out if it's an expression type that would benefit from further analysis, and then if so to delegate to a type-specific parser that knows how to produce nested hclwrite.Expression nodes based on the hclsyntax AST.

Earlier work already established that function call expressions, object constructor expressions, and tuple constructor expressions are worthy of special support when we added the interim TokensForFunctionCall, TokensForObject, and TokensForTuple functions. Therefore I expect I'd start by giving the expression parser specialized support for hclsyntax.FunctionCallExpr, hclsyntax.ObjectConsExpr, and hclsyntax.TypleConsExpr to continue that precedent.

However, the "loose end" in the current design is exactly what API would make sense for interacting with these specialized forms of hclsyntax.Expression. Each one would want to expose a different API for interrogating and modifying the expression, which leads to a similar situation as how HTML DOM represents different element types with different specializations of the "element" base type.

For the use-case given in this issue, at a high-level I'd expect to be able to:

• Ask the hclwrite.Expression if it's representing an object constructor, and if so to return an object-constructor-specific wrapper object.
• That wrapper object would then have methods similar to the ones on hclwrite.Body for manipulating the nested attributes. For example:
  • A RemoveAttribute("bar") method for deleting all of the tokens related to that attribute
  • An Attributes() method to get a map[string]*hclwrite.Attribute representing all of the currently-declared attributes , so that you can traverse into the expression for a specific nested attribute and perform all of the same operations in that nested context.

Perhaps then there would be hclwrite.FunctionCallExpr, hclwrite.ObjectConsExpr, and hclwrite.TupleConsExpr to match with the types of the same name in hclsyntax, and then hclwrite.Expression would have a method AsObjectCons() (ObjectConsExpr, ok) which returns an ObjectConsExpr representation of the same expression if and only if the expression is actually an object constructor expression; otherwise it would return the zero value of ObjectConsExpr and false.

That design would get pretty chaotic if the desired end state were to offer an hclwrite equivalent to every single hclsyntax.Expression implementation, but it's not clear to me that such an exhaustive design is actually required: use-cases discussed so far have typically needed only to manipulate the three main composite expression types I've discussed here, and for the rare case that needs more it would remain possible to manipulate raw tokens as we can already do today.

I don't have the time or motivation right now to work on any of this myself, but I hope the above is useful to someone else who might be interested in experimenting. Note that I'm not an HCL maintainer anymore, so if someone does want to work on this I'd suggest discussing your plans with the HCL maintainers first to make sure that such a contribution would be welcomed.