Question - Is ordering deterministic

[udf2457]

Could you clarify whether hcl ordering is deterministic ? As we know, go maps are not deterministic in their output (i.e. you will get a different order each time).

Therefore, taking the README example:

  process "main" {
    command = ["/usr/local/bin/awesome-app", "server"]
  }

  process "mgmt" {
    command = ["/usr/local/bin/awesome-app", "mgmt"]
  }

And the readme definition:

type ServiceConfig struct {
	Protocol   string          `hcl:"protocol,label"`
	Type       string          `hcl:"type,label"`
	ListenAddr string          `hcl:"listen_addr"`
	Processes  []ProcessConfig `hcl:"process,block"`
}

type ProcessConfig struct {
	Type    string   `hcl:"type,label"`
	Command []string `hcl:"command"`
}

Will the output ALWAYS be as defined in the hcl config file ? i.e. main then mgmt in this example.

The reason I am asking is because I am writing a playbook type Go app where the tasks need to be undertaken in a specific order otherwise Bad Things Happen (TM). I am looking around for config tooling and hcl looks like it might tick many boxes as long as its output is deterministic.

[apparentlymart]

Hi @udf2457,

Since HCL has multiple levels of abstraction at which you can decode this is not a straightforward answer in the general case, but since you are clearly using gohcl we can focus on that to start.

In the example you showed, gohcl will write the representations of your process blocks into ServiceConfig.Processes in the same order they appear in the source configuration file. So yes, the one with label "main" will always appear first when given this input.

The rest of this answer is just because I expect others will find this answer in future and might have a different situation, so I want to tell a fuller story.

The lowest-level API for decoding in HCL is the Body.Content method (and similar). That method takes a description of a schema, which is a set of valid attribute/argument names and a set of valid block types.

That method returns hcl.BodyContent, which includes a map representing the attributes and a slice representing the blocks. This means that at the lowest level attributes do not preserve definition order, but nested blocks do. In particular, the order preserved is the overall order of all blocks regardless of type, because they are all interleaved together into a single flat slice.

The higher-level abstractions on top of that API can at best preserve the detail the lowest-level API offers, but in practice they tend to be at least a little lossy in return for higher convenience. In particular:

• gohcl requires a separate Go struct field for each distinct nested block type name, and so it loses the information about the relative ordering between blocks of different types, and can only preserve the ordering of blocks of the same type.
• hcldec can decode blocks into a number of different cty.Type, chosen on a per block type basis. At minimum this is lossy in the same way gohcl is. If decoding into a map or object type then it will additionally discard the ordering even of blocks of the same type, and will instead only track the nested blocks by their unique labels. (and return an error if the labels are not unique)

Choosing which of the API levels to work at is a trade-off between convenience and flexibility. Only the lowest-level API can preserve the overall ordering of all blocks regardless of type, but in practice that hasn't been an issue for most HCL-based languages designed so far.