vitess
vitess copied to clipboard
Bug Report: When using MySQL without mysqld_safe a stale lock file prevents restarts
Overview of the Issue
When we run MySQL with mysqlctld
but without mysqld_safe
and MySQL has an OOM issue, it stops with leaving a lock file in place for the MySQL socket.
In case of mysqlctld
though, we know that we manage mysqld
and that we can safely restart, making it more robust against issues like OOM in this scenario.
Reproduction Steps
Run mysqld
with mysqlctld
but without mysqld_safe
. In case MySQL OOMs, it won't restart MySQL.
Binary Version
-
Operating System and Environment details
-
Log Fragments
I1119 20:13:57.325598 1 mysqld.go:388] Mysqld.Start(1700424834) stderr: 2023-11-19T20:13:57.325526Z 0 [ERROR] [MY-010119] [Server] Aborting
I1119 20:13:57.325590 1 mysqld.go:388] Mysqld.Start(1700424834) stderr: 2023-11-19T20:13:57.325512Z 0 [ERROR] [MY-010268] [Server] Unable to setup unix socket lock file.
I1119 20:13:57.325549 1 mysqld.go:388] Mysqld.Start(1700424834) stderr: 2023-11-19T20:13:57.325477Z 0 [ERROR] [MY-010259] [Server] Another process with pid 15 is using unix socket file.
Looks like https://github.com/vitessio/vitess/pull/14553 doesn't not properly fix this, as there can be a newline in the file and it then fails with:
E1124 14:11:50.284750 1 mysqld.go:450] Mysqld.Start(1700835110): error parsing pid from lock file: strconv.Atoi: parsing "162\n": invalid syntax
E1124 14:11:50.284993 1 mysqlctld.go:125] failed to start mysqld: strconv.Atoi: parsing "162\n": invalid syntax
Reopening this one as there's still cases where this can happen. One theory is that it could be that mysqlctld
gets assigned the same PID as in the lock file.
Additionally, we can improve the logging to see more information about the other process.