charts icon indicating copy to clipboard operation
charts copied to clipboard

[bitnami/mlflow] Upgrading 2.5.0- MLFlow chart to newer version fails with `The secret "mlflow-tracking" does not contain the key "flask-server-secret-key"`

Open ogusak opened this issue 7 months ago • 1 comments

Name and Version

bitnami/mlflow 2.5.1+

What architecture are you using?

amd64

What steps will reproduce the bug?

  1. MLFlow earlier than 2.5.1 deployed to a kubernetes cluster
  2. run helm upgrade to upgrade to the latest chart version (4.0.0)

helm deployment fails with error:

Error: UPGRADE FAILED: execution error at (mlflow/templates/tracking/auth-secret.yaml:22:30): 
PASSWORDS ERROR: The secret "mlflow-tracking" does not contain the key "flask-server-secret-key"

Are you using any custom parameters or values?

Parameters for database, etc provided via values.yaml file

What is the expected behavior?

Upgrade should succeed

What do you see instead?

helm deployment fails with error:

Error: UPGRADE FAILED: execution error at (mlflow/templates/tracking/auth-secret.yaml:22:30): 
PASSWORDS ERROR: The secret "mlflow-tracking" does not contain the key "flask-server-secret-key"

Additional information

This is caused by new key flask-server-secret-key added to auth-secret.yaml template in version 2.5.1

The secret is rendered with common.secrets.passwords.manage helper template which has the following parameter:

- failOnNew - Boolean - Optional - Default to true. If set to false, skip errors adding new keys to existing secrets.

In the chart this parameter is not provided

flask-server-secret-key: {{ include "common.secrets.passwords.manage" (dict "secret" (include "mlflow.v0.tracking.fullname" .) "key" "flask-server-secret-key" "providedValues" (list "tracking.auth.flaskServerSecretKey") "context" $) }}

and hence defaults to false and that is causing the error.

ogusak avatar Jun 06 '25 00:06 ogusak

Hi @ogusak ,

Sorry for the delay and thank you for bringing this issue to our attention. We appreciate your involvement! Since you are familiar with the problem, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance

dgomezleon avatar Jun 11 '25 14:06 dgomezleon

Hey, we faced similar issue, for us the fix was to remove the "mlflow-tracking" secret and then recreate it with the new version, for some reason I believe there was a validation before upgrading the k8s secret object, since this is a new parameter (flask-server-secret-key).

awedis avatar Jun 24 '25 09:06 awedis