consul with TLS does not read env vars set by -putenv

[teutat3s]

• what is happening and what you expect to see

This is in a SmartOS zone. Configured consul with containerpilot from the autopilotpattern and documented environment variables by HashiCorp for TLS:

$ export CONSUL_HTTP_ADDR=https://localhost:8501
$ export CONSUL_CACERT=consul-agent-ca.pem
$ export CONSUL_CLIENT_CERT=dc1-cli-consul-0.pem
$ export CONSUL_CLIENT_KEY=dc1-cli-consul-0-key.pem

These are set for containerpilot via containerpilot -putenv in the preStart() function of consul-manage.

I'd expect containerpilot to work with these env vars set like this, but instead I needed to do something like

svccfg -s containerpilot setenv CONSUL_CACERT "/ssl/ca.crt"
svccfg -s containerpilot setenv CONSUL_CLIENT_CERT "/ssl/cgn-1.crt"
svccfg -s containerpilot setenv CONSUL_CLIENT_KEY "/ssl/cgn-1.key"

or change the configuration file containerpilot.json5 with the following consul stanza to make TLS work.

consul: {
  address: "https://127.0.0.1:8501",
  tls: {
    cafile: "/ssl/ca.crt",               
    clientcert: "/ssl/cgn-1.crt",        
    clientkey: "/ssl/cgn-1.key",        
  }
  },
...

Otherwise the below error messages appeared.

Took me some time to figure this out... What would be the correct way to handle this?

• the output of containerpilot -version

3.8.0

• the ContainerPilot configuration you're using

{
  consul: "{{ if .CONSUL_ENCRYPT }}https://127.0.0.1:8501{{ else }}127.0.0.1:8500{{ end }}",
  logging: {
        level: "INFO",
        format: "default",
        output: "/var/log/containerpilot.log"
      },
  jobs: [
    {
      name: "preStart",
      exec: ["/usr/local/bin/consul-manage", "preStart"],
    },
    {
      name: "consul",
      port: {{ if .CONSUL_ENCRYPT }}8501{{ else }}8500{{ end }},
      {{ if .CONSUL_DEV }}exec: [
        "/usr/local/bin/consul", "agent",
        "-dev",
        "-config-dir=/opt/local/etc/consul"],
      {{ else }}exec: [
        "/usr/local/bin/consul", "agent",
        "-server",
        "-bootstrap-expect", "3",
        "-config-dir=/opt/local/etc/consul"{{ if .CONSUL_UI }},
        "-ui"{{ else }}{{ end }}],{{ end }}
      when: {
        source: "preStart",
        once: "exitSuccess"
      },
      health:{
        exec: ["/usr/local/bin/consul-manage", "health"],
        interval: 10,
        ttl: 25
      }
    },
    {
      name: "preStop",
      exec: ["/usr/local/bin/consul-manage", "preStop"],
      when: {
        source: "consul",
        once: "stopping"
      }
    }
  ]
}

• the output of any logs you can share; if you can it would be very helpful to turn on debug logging by adding logging: { level: "DEBUG"} to your ContainerPilot configuration.

service registration failed: Put https://127.0.0.1:8501/v1/agent/service/register: remote error: tls: bad certificate
service update TTL failed: Put https://127.0.0.1:8501/v1/agent/check/update/service:consul-vault-test: remote error: tls: bad certificate