[BUG] - Removing Keycloak initial_root_password from config causes Keycloak to be restarted on each deployment

[sblair-metrostar]

Describe the bug

Following the guidance in the Nebari documentation that says to remove the Keycloak initial_root_password after initial deployment should have no effect on the environment. However, the Nebari CLI appears to fill in this blank with a random password on each nebari deploy which results in a change to the helm chart and causes the keycloak-0 pod to be restarted. This doesn't appear to break anything but can cause upwards of a minute of downtime while Keycloak comes back up.

Expected behavior

Removing the initial_root_password key/value from nebari-config.yaml should not have any side effects on the environment.

OS and architecture in which you are running Nebari

Linux, x64

How to Reproduce the problem?

Change nebari-config.yaml after initial deployment.

Before:

security:
  keycloak:
    initial_root_password: "<mysupersecretpassword>"
  authentication:
    type: password

After:

security:
  authentication:
    type: password

Run nebari deploy -c nebari-config.yaml and terraform applies a change the Keycloak helm chart. When complete, check the age of the keycloak-0 pod to confirm it is new.

Command output

[terraform]: Terraform will perform the following actions:
[terraform]: 
[terraform]:   # module.kubernetes-keycloak-helm.helm_release.keycloak will be updated in-place
[terraform]:   ~ resource "helm_release" "keycloak" {
[terraform]:         id                         = "keycloak"
[terraform]:         name                       = "keycloak"
[terraform]:         # (27 unchanged attributes hidden)
[terraform]: 
[terraform]:       - set {
[terraform]:           - name  = "initial_root_password" -> null
[terraform]:           - value = "<mysupersecretpassword>" -> null
[terraform]:         }
[terraform]:       + set {
[terraform]:           + name  = "initial_root_password"
[terraform]:           + value = "<newrandompassword>"
[terraform]:         }
[terraform]: 
[terraform]:         # (1 unchanged block hidden)
[terraform]:     }
[terraform]: 
[terraform]: Plan: 0 to add, 1 to change, 0 to destroy.
[terraform]: module.kubernetes-keycloak-helm.helm_release.keycloak: Modifying... [id=keycloak]

Versions and dependencies used.

Nebari: 2023.11.1

Compute environment

AWS

Integrations

No response

Anything else?

No response

[Adam-D-Lewis]

Thanks for reporting @sblair-metrostar.

I think the recommendation for now would be to change your nebari-config.yaml to

security:
  keycloak:
    initial_root_password: ""  # <--- blank string here
  authentication:
    type: password

but this should be tested before trusting this advice.

I would support a PR to the docs to make this more clear or a PR to allow the behavior you initially expected. Chime in if you (or others) are interested in working on this issue.