Mattermost can't reach mysql

[matthew-parlette]

I've followed the instructions to install mattermost via operators on kubernetes, but my mattermost init pod 1 can never reach the mysql instance. I'm seeing this repeated in the logs:

init-check-mysql waiting for mysql
init-check-mysql   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
init-check-mysql                                  Dload  Upload   Total   Spent    Left  Speed
init-check-mysql   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (7) Failed to connect to db-3366d4-mysql-master.mattermost port 3306: Connection refused
init-check-mysql waiting for mysql

I've tested the connection to mysql by connecting to it from another pod to two different services in the mattermost namespace:

$ kubectl run -it --rm --image=mysql:5.6 --restart=Never mysql-client -n mattermost -- mysql -h db-3366d4-mysql-master.mattermost -p<password>

ERROR 2003 (HY000): Can't connect to MySQL server on 'db-3366d4-mysql-master.mattermost' (111)

$ kubectl run -it --rm --image=mysql:5.6 --restart=Never mysql-client -n mattermost -- mysql -h mysql.mattermost -p<password> 

mysql> 

But mattermost seems to want to connect to it through the db-3366d4-mysql-master service rather than the mysql service. Is there any way to specify this? Or is it something in the operator code that needs to be changed?

Let me know if you need any additional information from me, thanks!

[cpanato]

hello @matthew-parlette what version are you using?

[mterhar]

Mine was running fine for several months and then this problem happened randomly.

[mterhar]

Mine was running fine for several months and then this problem happened randomly.

I scaled the operator down to 0 pods and opened the deployment. Manually changed the db-33333-mysql.mattermost address to mysql and the services all connected back up...

Obviously this isn’t a fix. Just more info for troubleshooting.