how to convert HCL2 into json?

[shanye997]

hello! I used function unmarshal to convert HCL into interface{}, but I meet the error of "unknown token : data xxx xxx". I think it is the problem of HCL version. But I can not find the way to convert HCL2 into json. It seems hcldec cmd can do the work, but I cannot find the function which can do the same thing.

[apparentlymart]

Hi @shanye997,

There is no single function to "convert HCL to JSON", but based on your reference to hcldec I assume that what you mean is to decode an HCL configuration into a data structure that you can then serialize as JSON, since that's what the hcldec tool does.

The hcldec tool is implemented in terms of the HCL module API, so there's nothing it's doing that you cannot also do directly using library calls. However, the hcldec tool does have some extra complexity for decoding its own special language for describing the "spec" for decoding, which you wouldn't need when making a library call because you can instantiate hcldec.Spec values directly in your calling code.

The high-level steps would be:

• Write out a tree of hcldec.Spec values that describes how to map from your HCL-based language to the JSON structure you'd like to produce from it.
• Parse and decode your user's HCL input into a hcl.Body object.
• Use hcldec.Decode with the spec and body created in the previous steps to produce a value, represented using the cty.Value type from the third-party library cty.
• Use cty's JSON encoder to serialize that value as JSON.

Here's an example for a relatively-simple HCL-based language that just includes two top-level arguments named a and b:

// This spec describes the result being an object with "a" and "b" properties,
// each of which is populated from an HCL argument (attribute) of the same
// name. The names inside the `AttrSpec` objects are the names to expect
// in the input HCL file.
spec := hcldec.ObjectSpec{
    "a": hcldec.AttrSpec{
        Name: "a",
        Type: cty.String,
    },
    "b": hcldec.AttrSpec{
        Name: "a",
        Type: cty.String,
    },
}

// This will try to parse HCL source code from src, producing a
// file object that has a Body field of type hcl.Body representing
// the top-level content of the file.
file, diags := hclsyntax.ParseConfig(src, "example.hcl", hcl.InitialPos)
if diags.HasErrors() {
    // (handle the errors and halt)
}

// Now we can ask hcldec to decode the loaded body using the
// spec created earlier.
v, moreDiags := hcldec.Decode(file.Body, spec, nil)
diags = append(diags, moreDiags...)
if moreDiags.HasErrors() {
    // (handle the errors and halt)
}

// v is now a cty.Value value whose type is guaranteed to be
// an object type with "a" and "b" attributes, because that's
// what the spec described. You can serialize that to
// JSON using cty's own JSON package.
// (Note: "json" here is "github.com/zclconf/go-cty/cty/json",
// not "encoding/json" from the Go standard library.)
result, err := json.Marshal(v, v.Type())
if err != nil {
    // (handle the error and halt)
}

// Now "result" is a []byte containing the JSON representation
// of the decoded object.

The above is the essence of what the hcldec tool does, but with the spec hard-coded instead of loaded dynamically from a configuration file. hcldec also supports specifying variables that are made available during evaluation which I didn't include above, but could be done by setting the third argument of hcldec.Decode whereas I just passed nil to represent that no variables or functions are available.

[shanye997]

Thank you for your reply! That helps me a lot. I wonder when I am not sure about the structure of hcldec.Spec, how can I decode an HCL configuration into it? I need to decode a lot of different HCL configurations.

[apparentlymart]

Hi @shanye997,

The hcldec.Spec values are one way to describe to HCL the schema of the configuration language you are (presumably) designing. I cannot tell you exactly how to write your spec because I don't know the schema of your language.

[shanye997]

Okay! I'm actually find a function which can do the same thing as hcl.Unmarshal([]byte(hclStr), &v). unmarshal a HCL configuration to a interface{}. Does hcl2 have the same function?

[shanye997]

hi, I just find the actual problem I meet. When I use quotes or functions in HCL configuration, not hard-coded value. The hcl.Unmarshal will return an unknown token error. Is there any parameters can treat all values as hard-coded and solve my problem?

[apparentlymart]

Hi @shanye997,

Unfortunately I'm not sure what exactly you are trying and what's not working because you seem to be skipping details and I cannot see what you are trying and I don't yet really even understand what your final goal is.

It sounds like you might now be trying to use the legacy version of HCL -- major version 1 -- but you've seen an error because you are trying to work with a configuration file that was written for HCL 2. HCL 2 has many new features that HCL 1 did not support, so you will not be able to use the old version unless you are intending to implement a language that is based on that old version.

[genelet]

May this article help? https://medium.com/@peterbi_91340/marshal-and-unmarshal-hcl-files-1-3-d7591259a8d6

[shanye997]

It's helpful, thank you! I wonder if there is any way to unmarshal HCL configuration without defining the struct? I want to unmarshal the HCL configuration to map[string]interface{} struct.

[genelet]

It's helpful, thank you! I wonder if there is any way to unmarshal HCL configuration without defining the struct? I want to unmarshal the HCL configuration to map[string]interface{} struct.

As mentioned in the above discussions, this is usually impossible because almost all HCL configuration files are defined according to objects (= go struct) in your projects. You have to specify target in hclsimple.Decode or dethcl.Unmarshal when unmarshalling.

There are CLIs hcl2json and json2hcl here which works under condition that there is no go struct but just simple map and list.