Can't use variables or locals on block labels

[ivantopo]

trafficstars

Nomad version

Nomad v1.0.0-rc1 (afea734825ebaade422bbf2d31364abff6def928)

Operating system and Environment details

Ubuntu 20.04.1 LTS

Issue

I was trying to use locals/vars to customize the name of a job, following the example in the docs but I'm getting this error:

Error getting job struct: Error parsing job file from locals-bug.nomad:
locals-bug.nomad:10,6-8: Invalid string literal; Template sequences are not allowed in this string. To include a literal "$", double it (as "$$") to escape it., and 2 other diagnostic(s)

Reproduction steps

• Define variables and/or locals in a job file
• Try to interpolate any variable and/or locals on the job name label.

Job file (if appropriate)

variables {
  name = "app"
  suffix = "cluster"
}

locals {
  job_name = "${var.name}_${var.suffix}"
}

job "${local.job_name}" {
  datacenters = [ "FSN" ]

  group "app-nodes" {
    count = 3

    task "app-node" {
      driver = "docker"

      config {
        network_mode = "host"
        image = "hello-world:latest"
      }
    }
  }
}

[notnoop]

Thank @ivantopo for highlighting it. It's indeed a limitation that we want to eliminate in next few releases. We'll update the docs to call this out for now.

For context, the limitation is built into HCL2 parser: https://github.com/hashicorp/hcl/blob/v2.7.2/hclsyntax/parser.go#L1637-L1638 ; and we are discussing internally removing it and how it may affect other products using HCL2.

[ivantopo]

Thanks for the insights, @notnoop! I'll stay alert for any updates and move on without these interpolations in the meantime.

[tgross]

@ivantopo by the way, there is a workaround if you're willing to have dynamic blocks in your jobspec. See https://www.nomadproject.io/docs/job-specification/volume#volume-interpolation where we have an example for volume. The labels field is what gets used for the generated volume block's label.

[sofixa]

@tgross that doesn't work with the "job" stanza:

dynamic "job" {
    for_each = var.job_name
    labels = [var.job_name]
    content {
...

x.nomad:21,1-8: Unsupported block type; Blocks of type "dynamic" are not expected here.

The only way to have dynamic job names seems to be to pass through levant

[traceypooh]

but. not to be too uppity... (divulges cards, being uppity...)

the only thing not dynamic / interpolation friendly AFAICT is the job "[[NAME]]" and that seems like... a shame (rhymes)

we loved levant - but aren't going to use it anymore for just a single string swapout, eg: sed -i "s/NOMAD_VAR_SLUG/$NOMAD_VAR_SLUG/" project.hcl https://gitlab.com/internetarchive/nomad/-/blob/master/.gitlab-ci.yml#L52

here's the project.nomad / project.hcl general template for GitLab + Nomad: https://gitlab.com/internetarchive/nomad/-/blob/master/project.nomad#L118

there's just that single string substitution needed now.

can live with sed - but if we all didn't have to.... 🙌

[traceypooh]

perhaps worth pointing out (am sure it's known) that you can't use dynamic blocks inside docker driver config. not the end of the world and can workaround. but perhaps worth mentioning.

i'm super chuffed w/ hcl v2 overall - it's super powerful and great. thanks for the behind the scenes wizardry getting that into nomad.

[tgross]

the only thing not dynamic / interpolation friendly AFAICT is the job "[[NAME]]" and that seems like... a shame (rhymes)

Yeah, totally agree with you here. We even have to do silly things in our end-to-end testing to work around this. I don't think there's anything inherent to HCL2 that prevents block labels from being interpolated, it's a matter of spending some time to rework how we're doing the decoding a little bit. Definitely will get on the roadmap.

perhaps worth pointing out (am sure it's known) that you can't use dynamic blocks inside docker driver config. not the end of the world and can workaround. but perhaps worth mentioning.

Thanks! That also got reported in https://github.com/hashicorp/nomad/issues/9871 and the fix will ship in the upcoming Nomad 1.0.4 https://github.com/hashicorp/nomad/issues/9921

[traceypooh]

thanks @tgross !

[pscheit]

since this isnt added right now, this example in the docs is very confusing:

https://www.nomadproject.io/docs/job-specification/hcl2/locals#examples

[JanMa]

Hey,

can you give an estimate if this will be fixed in the next release(s)? Being able to set dynamic Job, Task and Group names using HCL2 would allow me to write generic Job templates without the need for external tooling. Currently I always need some additional tooling to render a job template before passing it to Nomad. Using only the built-in features would simplify my deployment pipelines quite a bit :-)

[nepeat]

I would like to note that this is especially an issue with container labels, both quoted and unquoted.

job "sample" {
    datacenters = ["dc1"]
    group "redacted_group" {
        task "redacted_task" {
            driver = "docker"

            config {
                labels {
                    traefik.enable = "true"
                }
                image = "nginx:1.19.10-alpine"
            }
        }
    }
}
Error getting job struct: Failed to parse using HCL 2. Use the HCL 1 parser with `nomad run -hcl1`, or address the following issues:
sample:9,21-28: Argument or block definition required; An argument or block definition is required here. To set an argument, use the equals sign "=" to introduce the argument value.

job "sample" {
    datacenters = ["dc1"]
    group "redacted_group" {
        task "redacted_task" {
            driver = "docker"

            config {
                labels {
                    "traefik.enable" = "true"
                }
                image = "nginx:1.19.10-alpine"
            }
        }
    }
}
Error getting job struct: Failed to parse using HCL 2. Use the HCL 1 parser with `nomad run -hcl1`, or address the following issues:
sample:9,21-22: Invalid argument name; Argument names must not be quoted.

[lemon-li]

version: 1.0.4 I was trying to HCL2 example , but I'm getting this error:

Error getting job struct: Error parsing job file from foobar.nomad:
foobar.nomad:12,6-8: Invalid string literal; Template sequences are not allowed in this string. To include a literal "$", double it (as "$$") to escape it.
foobar.nomad:4,31-43: Unsupported attribute; This object does not have an attribute named "name_prefix".

[pznamensky]

Also tried to use local variables and use locals example from the documentation, but it still doesn't work.

~ $ nomad plan example.nomad 
Error getting job struct: Error parsing job file from example.nomad:
example.nomad:4,31-43: Unsupported attribute; This object does not have an attribute named "name_prefix".

~ $ nomad version
Nomad v1.1.3 (8c0c8140997329136971e66e4c2337dfcf932692)

job file:

# A computed default name prefix
locals {
  default_name_prefix = "${var.project_name}-web"
  name_prefix         = "${var.name_prefix != "" ? var.name_prefix : local.default_name_prefix}"

  # unlike variables, locals don't have type constraints, so if you use
  # functions that take maps but not objects, you may need to convert them
  number_of_ports = length(convert({"www" = "80"}, map(string)))
}

job "example" {

  name = "${local.name_prefix}_loadbalancer"

  datacenters = ["dc1"]

  group "cache" {
    network {
      port "db" {
        to = 6379
      }
    }

    task "redis" {
      driver = "docker"

      config {
        image = "redis:3.2"

        ports = ["db"]
      }

      resources {
        cpu    = 500
        memory = 256
      }
    }
  }
}

[xeroc]

We are currently using sed, but as noted also by others in this thread, we don't really like this solution.

[zhujinhe]

We are also using sed right now, looking for a more native way to do this.

job "foo" {
  name = var.bar
}

btw, it will create a job with "foo" as it's ID and the value of bar as it's Name. If you modify the value of bar, the job with ID eques foo will be update in-place, NO new job will be created.

[vvarga007]

Any update on this? Three years later, there is still no solution.

[Yethal]

If variables in block labels are a nogo can we at least get the option of using an unlabeled job block and specifying the id the same way we can specify the name?

[hashworks]

It might make sense to allow a syntax like job { and expect a job id definition using CLI flags when one runs nomad run?

[josvegit]

this basically makes nomad-pack useless XD

[tgross]

@josvegit except that Pack is able to interpolate HCL labels? There's examples in the community registry that demonstrate this exactly: https://github.com/hashicorp/nomad-pack-community-registry/blob/main/packs/democratic_csi_nfs/templates/controller.nomad.tpl#L1

Label interpolation just isn't available in Nomad natively (and as noted, it's a HCL2 parser issue, so it's non-trivial to fix without making changes that potentially impact all other HashiCorp products). If you're finding otherwise, that's probably a Pack bug. And in any case, the editorization without detail isn't particularly productive.

[Yethal]

@tgross Would it be possible to interpolate the job label by using a dynamic block to construct the job block? EDIT: nvm I see this is not possible