support for kubernetes secrets type in auth_modules for Multi-Target Support (BETA)

[ninad-opsverse]

I am currently using the Multi-Target Support feature in the postgres_exporter to scrape metrics from multiple Postgres instances deployed in my Kubernetes cluster. The exporter is configured using a scrape job as mentioned in the Muti-Target Support feature docs.

The Muti-Target Support feature currently supports preconfigured auth_modules as way of authenticating to the postgres instances. But the auth_modules only supports the userpass type. I would like to request the support for Kubernetes secrets in the auth_module, allowing users to securely store and manage database credentials in Kubernetes secrets for multiple Postgres instances rather than hardcoding user and password values in the configuration..

Potential Implementation

• Allow the auth_module to reference a Kubernetes secret by name.

Versions

Postgres Exporter - v0.15.0 Kubernetes - v1.28.4

Thank you for considering this feature request!

[sysadmind]

I don't know that adding in direct kubernetes integration is a great direction from a maintainability perspecitve. What about referencing the secrets that are mounted to the pod as a volume? I think adding a file reference for username/password would be a more robust option and would benefit more users

[ninad-opsverse]

Hey @sysadmind, thanks for the reply.

I wanted to confirm if referencing secrets mounted to the pod as volume, is this supported as of today or needs to be built? And does the secrets that you mentioned which are mounted to the pod as a volume get created by default or do we have to manually create and mount them to the pod?

[sysadmind]

Using a file for username/password is not yet supported but seems like a generally useful feature so I think we would be happy to add it. How secrets get mounted to the pod is beyond the scope of this exporter. That would be for you to determine.

[xfiodembo]

auth_modules is fine, it would be nice if it always read the password from the file so that we can rotate it or use short lived tokens without having to restart the application.

[PIrojahPerbak]

Just following up on this, any updates on the file support

[navinpai]

@sysadmind What would a probable solution look like? Something like:

# postgres_exporter.yml
additional_auth_modules: "auth.yml"
auth_modules: {}

And then have

# auth.yml
auth_modules:
  foo1: # Set this to any name you want
    type: userpass
    userpass:
      username: first
      password: firstpass
    options:
      # options become key=value parameters of the DSN
      sslmode: disable

? Or should the auth.yml file have a different format? Thoughts?

[itaizelther]

I think what might fit here is the solution grafana provides for secrets in their configuration file (grafana.ini). Maybe upon configuring the target and reading the password, which probably happens here: https://github.com/prometheus-community/postgres_exporter/blob/fca2ad84cd79fea6b414a2ce21fe8ee5b8d5a5a4/config/config.go#L111 we check for certain syntax and read from a file which is a secret mount?

Something like this:

auth_modules:
  client.app:
    type: userpass
    userpass:
      username: admin
      password: "__file{/path/to/secret/mount}"

I think it will be more convenient to enable to create a file for each distinct secret, and still maintain the auth_modules tree in the helm values.

[gregkoganvmm]

Referencing an environment variable may be an easier lift, but unless there is a good way to inject credentials, the whole file needs to be stored as a Kubernetes secret (or be otherwise secured). This makes source control of the config more challenging and if you add custom queries, it may become a problem for larger setups. Basically, any way to inject credentials into the config would make securing db connections much better.