Authentication failure on one pod after cluster upgrade from 3.3.6.9 to 3.3.7.5

[NephilimJaeger]

Bug Description

After upgrading our TDengine cluster from version 3.3.6.9 to 3.3.7.5, the cluster runs normally for a period of time, but then one of the pods starts experiencing authentication failures when attempting to send commands to the mnode using the default password. The command should use the actual database password, not the default.

To Reproduce Steps to reproduce the behavior:

1. Upgrade TDengine cluster from version 3.3.6.9 to 3.3.7.5
2. Cluster operates normally for a few hours
3. One pod begins failing authentication when connecting to mnode using default credentials

Expected Behavior All pods should maintain consistent authentication after the upgrade, given that no password changes were made during or after the upgrade process.

Actual Behaviour One pod fails authentication when trying to connect to mnode The password was not explicitly changed during or after the upgrade Eventually, only the mnode pod continue working normally Error appears spontaneously after the cluster has been running for some time

Screenshots Pod Logs

Environment (please complete the following information):

TDengine Version (before upgrade): 3.3.6.9 TDengine Version (after upgrade): 3.3.7.5 Deployment: Kubernetes cluster Platform: Linux

[mesquita]

Very important!

[kailixu]

@NephilimJaeger Your bug report states that "The command should use the actual database password, not the default", but the reproduction steps say a pod fails "using default credentials (i.e., 'taosdata')". This is confusing. Could you please clarify:

1. Was the default 'root' password changed in version 3.3.6.9? 2) If it was changed in 3.3.6.9, why would a pod try to connect using the default password 'taosdata' after the upgrade to 3.3.7.5?

[NephilimJaeger]

1.) Yes, the root default password was changed in a previous version. 2.) Exactly. It should not try to use the default password, and that is the bug. I don't know why the pod is trying to run a command using the default password when the password was already changed before.

@NephilimJaeger Your bug report states that "The command should use the actual database password, not the default", but the reproduction steps say a pod fails "using default credentials (i.e., 'taosdata')". This is confusing. Could you please clarify:

1. Was the default 'root' password changed in version 3.3.6.9? 2) If it was changed in 3.3.6.9, why would a pod try to connect using the default password 'taosdata' after the upgrade to 3.3.7.5?

[NephilimJaeger]

I fixed the problem by setting the password as the TAOS_ROOT_PASSWORD env var, but the initialization script shows the password in the logs!!! I presume that command comes from a script that runs in the Docker image, maybe? Considering that the env var is passed to the image.

[kailixu]

@NephilimJaeger

1. The method you used to solve the problem is a workaround and this scenario has already been described in FAQ.
2. If the root password changed, it should not need to set the env to complete the the reboot procedure.
3. Additionally, passwords should never be displayed in the log..

We will address this issue in a future release.

Thanks a lot for your feedback!

@CC @zitsen @feici02 @sheyanjie-taos @boxiaobj @hzcheng

[zitsen]

1. We recommend you change root password at startup/init by env TAOS_ROOT_PASSWORD=xxxxxxx
2. If you have modified by other ways, please change the docker env, touch a file in your dataDir: .docker-entrypoint-root-password-changed and restart