Password containing `$` doesn't work when passed via `${env:DB_PASSWORD}`

[wand2016]

Component(s)

receiver/postgresql

What happened?

Password containing $ doesn't work when passed via ${env:DB_PASSWORD}. When passed directly, it works.

Description

Steps to Reproduce

1. Prepare PostgreSQL DB:
   • username: user
   • password: p@s$w0rd
2. Configure postgresql receiver:

  postgresql:
    ...
    username: "${env:DB_USERNAME}"
    password: "${env:DB_PASSWORD}"

3. Pass environment variables to OpenTelemetry CR:

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
...
spec:
  config: <configuration mentioned above>
  env:
    - name: DB_USERNAME
      value: user
    - name: DB_PASSWORD
      value: p@s$w0rd

Expected Result

Authentication passes.

Actual Result

Authentication fails.

Given password directly, authentication passes.

  postgresql:
    ...
    username: "${env:DB_USERNAME}"
    password: "p@s$$w0rd"

Collector version

otel/opentelemetry-collector-contrib:0.81.0

Environment information

Environment

contrib Docker Image

OpenTelemetry Collector configuration

No response

Log output

No response

Additional context

No response

[github-actions[bot]]

Pinging code owners:

• receiver/postgresql: @djaglowski

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[mx-psi]

The Collector supports recursively expanding ${env:...} and the legacy $ENV syntax in the contents of an environment variable, so you need to escape the $ as documented on the official docs:

Use $$ to indicate a literal $.

In your case this would be p@s$$w0rd

[wand2016]

Thanks for replying!

Actually, the password is stored in a Secret resource:

apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
...
spec:
  config: <configuration mentioned above>
  env:
    - name: DB_PASSWORD   
      valueFrom:
        secretKeyRef:
          name: db-secrets
          key: password

(By the way, the Secret resource is synchronized by External Secrets Operator in my project.)

Must it be stored in the Secret resource (or its secret backend) as escaped p@s$$w0rd?
This couples the secrets and opentelemetry collector, which makes reuse hard.
Is there any way to disable recursive expanding?

[mx-psi]

I moved this to opentelemetry-collector since this is not specific to the postgresql receiver, but rather applies to configuration more generally.

I think this would not happen if the expandconverter was not enabled. Since this is a legacy component that we want to remove in favor of ${env:...} syntax, we could add a feature gate to disable the expand converter and keep it at alpha for now.

@open-telemetry/collector-maintainers what do you think? Would you be open to such feature gate?

[atoulme]

Discussed in SIG 12/20, we will work towards deprecating the naked $ notation and support ${...} instead, and therefore avoid this type of bug moving forward.

See https://github.com/open-telemetry/opentelemetry-specification/pull/3744 for the spec change.

As part of this, we will probably deprecate the $ expand, as noted by Pablo, we can't really log a warning right now when this happens because the expansion happens before logging is set up. We can look into that.

[TylerHelmuth]

We've made progress toward this issue. The collector now:

1. supports logging during confmap resolution
2. prints a log if $ENV syntax is used.

To officially work towards deprecating the $ENV syntax I believe we do need a feature gate that can move use through:

1. allowing the syntax, with the option to disallow (alpha)
2. disallow the syntax, with an option to allow (beta)
3. disallow the syntax entirely (stable)

[TylerHelmuth]

Actually this issue is still blocked by https://github.com/open-telemetry/opentelemetry-collector/issues/7111