Enhancement: control Vault path if doesn't exists

[mjimeneznet]

Consul Template version

consul-template v0.25.0 (99efa642)

Configuration

Config: test.conf

consul {
  address = "http://consul.internal"
  ssl {
    enabled = false
  }
}
vault {
  address = "https://vault.internal"
}
template {
  source = "test.ctmpl"
  destination = "test.txt"
  error_on_missing_key = true
  left_delimiter  = "[{"
  right_delimiter = "}]"
}

Template: test.ctmpl

[{ range ls "webservice/frontend/env_vars" -}]
[{   scratch.MapSet "vars" .Key .Value -}]
[{ end -}]

[{ with secret "kv/webservice/frontend/env_vars" -}]
[{   range $k, $v := .Data.data -}]
[{     scratch.MapSet "vars" $k $v -}]
[{   end -}]
[{ end -}]

[{ range $k, $v := scratch.Get "vars" }]
[{ $k }]=[{ $v -}]
[{ end }]

Command

consul-template -config=test.conf -once -dry

Expected behavior

This is not a bug, I'm asking for an improvement.

Given the previous template, when asking consul, if the CONSUL PATH doesn't exist the execution continues. Also I can add a keyExists and control the output.

But, if the VAULT PATH doesn't exists I got the error described in this issue and the execution stalls. I would like to have the same behaviour as Consul with keyExists or something similar, to control it.

Why? Sometimes we don't need vars from VAULT, only from CONSUL. We create custom AMIs with Packer and we use this template for all instances. So it depends on the instance we spawn it might need values from Vault or not. So basically the behaviour we would like to have is:

if vaultKeyExists
  key_from_vault
else
  foo=bar
end 

In short, have the same control in vault that we have in consul when a path doesn't exists.

Actual behavior

2020/04/29 15:01:28.126705 [WARN] (view) vault.read(kv/webservice/frontend/env_vars): no secret exists at kv/data/webservice/frontend/env_vars (retry attempt 3 after "1s")
``

[mjimeneznet]

Later I've seen two more "issues" related to this one: #776 and #942

And I did a workaround to "fix" this:

{{ if secrets kv/metadata/endpoint | contains my_var_im_looking_for }}
{{   with secret kv/endpoint/my_var_im_looking_for }}
{{ /* DO THINGS */ }}
{{   end }}
{{ end }}

But I prefer to have something native to consul-template than a workaround.

[herrbpl]

Later I've seen two more "issues" related to this one: #776 and #942

And I did a workaround to "fix" this:

{{ if secrets kv/metadata/endpoint | contains my_var_im_looking_for }}
{{   with secret kv/endpoint/my_var_im_looking_for }}
{{ /* DO THINGS */ }}
{{   end }}
{{ end }}

But I prefer to have something native to consul-template than a workaround.

This also does not work when there is secret but its current version is deleted. Either "secrets" should not list deleted items or it should be possible to retrieve at least deleted secret metadata.

Siim

[josegonzalez]

I would also love this sort of functionality - if secret_exists $SECRET_PATH would allow us to make secrets optional. We have a ton of services that don't need secrets, but our platform requires that all of that be setup prior to deployment.