Converting Nomad Job Spec HCL to JSON and Back?

[nickpoulos]

It has been very difficult to prop up some basic tooling around HCL based config files.

There are multiple projects that try to help with this issue but I have not been able to find one to do this correctly for my use case. I am hoping somebody might be able to point me in the right direction.

I have tried:

1. Using the nomad cli.

This would be the ideal way except for a few things.

• It wants to know the values of vars ahead of time so it can interpolate them. I need to keep the variable names and ${} syntax in place.

• There seems to be no way to go back to HCL once we have the JSON

2. Using a tool like https://github.com/hashicorp/hcl/tree/main/cmd/hcldec

This would also be ideal, but one big exception:

• It requires the creation of a spec of its own to compile down to the correct 'flavor' of HCL in JSON format. (I think?)
• I could not find any official specs for HashiCorp HCL flavors like Nomad/Terraform and I imagine it would be an effort for me to take on myself. https://hcl.readthedocs.io/en/latest/go_decoding_hcldec.html

3. Using third party tools like https://github.com/tmccombs/hcl2json.

This particular ones seems to work well...almost.

• It leaves my variables and their related syntax alone so that is a +1.

• It does not quite generate the right JSON format for Nomad specs. Ex. Constraints, Templates (which are key for me)

• There is no way to go back to HCL after

4. Another third party tool - https://github.com/kvz/json2hcl.

• This one can go to from JSON to HCL and reverse.
• Does not support HCL2 (dealbreaker)

After digging into the code a little I found https://github.com/hashicorp/nomad/blob/main/jobspec/parse_job.go but could not find where it runs or how I might be able to use it.

Does anybody have any advice? Or has taken a stab at writing a Nomad hcldec spec file? Does hcldec do the reverse conversion?

Thanks

[apparentlymart]

Hi @nickpoulos,

The JSON variant of HCL relies on application-defined schema to resolve ambiguity inherent in the simpler JSON syntax (as compared to the HCL native syntax) so there is no general answer to "convert between native syntax and JSON syntax". Instead, this will always need to be an application-specific implementation that is aware of the application's own schema.

There are some third-party tools (which you've found) which try to do this generically, in spite of the caveats, and they can broadly work as long as the application is using a schema that matches each tool's assumptions about what a "generic schema" might look like. Even then, the conversion is inherently one-way -- native syntax to JSON -- because the JSON syntax is "lossy" and not self-describing enough to convert to native syntax without reference to a schema.

With all of that said then, I would agree with you that this seems like Nomad-specific format conversion seems like a lot of work to take on as a third-party individual. If you have a use-case for such a conversion then it may be better to share that use-case with the Nomad team and see if they would either address it by maintaining such a conversion tool themselves or by devising some other solution to meet your underlying problem.

As things exist today though, I believe Nomad accepts both the native syntax and JSON syntax directly, with the assumption that you'll just use whichever one makes sense for each particular case. I work mainly on Terraform so I'm not sure if there are Nomad-specific constraints here that make that tougher, but even so I think it would be better to start this discussion in the Nomad repository so it can be frame in terms of your Nomad use-case rather than in terms of potential technical solutions.

[flurie]

Would it be possible to do a functional 1:1 mapping if the export to/import from JSON included schema/annotation information that facilitated the conversion? If I understand correctly, this would require some reserved keywords that couldn't be present in any HCL, and it would require that, at a minimum, either applications that support "annotated" HCL-JSON be able to ignore these, or they be pruned prior to use?

I think there are two separate problems that get tangled up in this.

• The first is that it would be useful to be able to convert to and from "generic" HCL.
• The second is that specific applications and plugins for those applications can specify additional schema on top of HCL, and only a subset of valid HCL is also valid application code.

If we treat these problems as one problem, it's easy to punt and say that converters must be able to satisfy the constraints of the application consuming HCL; after all, the HCL spec will never be able to know all schemas in advance. However, I think targeting just the first problem is both feasible and beneficial, and we can leave validation of specific schemas up to each application. Should I write this up as a separate issue?

[apparentlymart]

Hi @flurie,

As we currently stand, there really is no such thing as "generic HCL". HCL is defined as a set of building blocks that languages can be defined in terms of, not as a self-contained language it its own right.

I think you are proposing to create a third HCL syntax that is built from JSON grammar but intentionally more verbose in order to describe at least as much structure as the native syntax can describe without any ambiguity about what the author intends a particular construct to mean. I would agree that is possible in principle but it sounds like a lot of work and would necessarily create something that isn't compatible with today's HCL JSON, and so applications would need some way to determine which JSON syntax the configuration author intends to use.

I think what I find most challenging about this issue is that it seems to be a classic "XY Problem". The problem statement is "I want to write some tools and so I need to work with HCL in JSON", but that doesn't really follow: the expected way to build tools is to interact with the HCL API to extract the information needed; converting to JSON or back should never be needed. But since this issue doesn't identify exactly what use-case this tool or tools would address, it's hard to suggest how to solve it using HCL as it exists or to identify specific new API features that would help address it within the existing abstraction. If there's some specific transformation or analysis to be done here then I'd prefer to talk about that, rather than talk about trying to achieve it by converting between syntaxes, which will always be lossy and therefore inappropriate for any tool which modifies configuration written by human authors.

[schmichael]

On the Nomad side this is still unresolved, but now somewhat intentionally instead of just by pure accident.

My solution was not to resolve the canonical HCL-JSON specification for jobspecs but to focus on improving the support for our much more broadly used API-JSON format. I updated the relevant docs here: https://github.com/hashicorp/nomad/pull/12766

I also added a -json flag to nomad job run which accepts the API-JSON format (well, "formats", there are sadly enveloped and plain variations): https://github.com/hashicorp/nomad/pull/12591

That -json flag matches what we do elsewhere with namspaces and quotas: HCL is accepted, but a -json flag accepts the API-JSON formatted JSON. Nowhere is the HCL-JSON format explicitly documented, specified, supported, or tested.

It's not a great state of affairs, but unless somebody has a strong reason to use HCL-JSON over the API-JSON format I don't think the Nomad team will invest time in trying to improve our HCL-JSON story.

Sorry! I know it would have been nice for there to be One True JSON format for Nomad jobspecs, but I don't think we can do that without just dropping the existing informal HCL-JSON as removing API-JSON isn't an option.