opt-in to automatically sourcing accessing `nomad/jobs` prefix variables

[tgross]

Many workloads will want to access all the variables available to them under the nomad/jobs/ prefix as environment variables. We can expect there to be a fairly common template block that gets copied over and over again by users to do that. Rather than forcing them to write out a template block, could we have a flag that opts-in the task to get all its accessible nomad/jobs variables sourced-in as environment variables? If so, what would this look like in implementation?

Suggested in https://github.com/hashicorp/nomad/issues/14996#issuecomment-1290103480 and opening for discussion here. cc @angrycub and @mikenomitch who might have some thoughts.

[ahjohannessen]

@tgross I have a rather big job for staging and dev that contains several groups. Avoiding to repeat the following would be awesome 👍

      template {
        destination = "${NOMAD_SECRETS_DIR}/nomad-variables.env"
        env         = true
        data        = <<EOT
{{- range nomadVarListSafe }}
  {{- if nomadVarExists .Path }}
    {{- with nomadVar .Path }}
      {{- range .Tuples }}
{{ .K | toUpper }}={{ .V | toJSON }}
      {{- end }}
    {{- end }}
  {{- end }}
{{- end }}
EOT
      }

I noticed that if no variables exist at job level, then an error is returned:

Missing: nomad.var.list(@default.global)

However, for groups or tasks in same job that have variables no error happens.

Additionally, when removing a variable at job level, the templating does not pick this up. I Need to restart the whole job to get things changes to kick in.

[blmhemu]

@ahjohannessen Upgrading to 1.4.2 might help. See https://github.com/hashicorp/nomad/issues/14996

[ahjohannessen]

@blmhemu I am on 1.4.2

[mikenomitch]

I am +1 on adding some sort of feature like this.

If you have job "foo", group "bar", and task "baz" do we want this to happen automatically for nomad/jobs/foo/bar/baz, nomad/jobs/foo/bar, and nomad/jobs/foo? What if you have a policy that also grants access to path some/random/vars?

Maybe instead of it automatically grabbing all the paths you can just specify the path in the template block. Something like:

template {
    destination = "${NOMAD_SECRETS_DIR}/my.env"
    env         = true
    var-source = "nomad/jobs/foo/bar/baz"
}

or maybe

template {
    destination = "${NOMAD_SECRETS_DIR}/my.env"
    env         = true
    var-source-paths = ["nomad/jobs/foo/bar/baz", "nomad/jobs/foo/bar", "some/random/vars"]
}

[blmhemu]

The template provided by @ahjohannessen works for some/random/path as well right ? Not just nomad/job/* ?

template {
    destination = "${NOMAD_SECRETS_DIR}/my.env"
    env         = true
    var-source-paths = ["nomad/jobs/foo/bar/baz", "nomad/jobs/foo/bar", "some/random/vars"]
}

The above suggestion looks good as well. Can prolly be simplified to below (ideally we do not need the dest. but would be good for debugging).

template {
    destination = "${NOMAD_SECRETS_DIR}/my.env"
    var-source-paths = ["nomad/jobs/*", "some/random/vars"]
}

[ahjohannessen]

@mikenomitch @blmhemu Great suggestion 👍 var-source-paths is not very descriptive, IMO. Perhaps like this:

template {
  destination = "${NOMAD_SECRETS_DIR}/nomad-variables.env"
  source_nomad_vars: ["nomad/jobs/*", "some/random/vars"]
}

destination could default to ${NOMAD_SECRETS_DIR}/nomad-variables.env if not defined and source_nomad_vars is defined.