Bug Report: Keyspace-Level Image Settings Issue in Vitess Operator
Bug Report: Keyspace-Level Image Settings Issue in Vitess Operator
Description
We’ve identified a bug in Vitess Operator v18 related to keyspace-level image settings. The issue occurs when keyspaces are sorted alphabetically. If one keyspace is configured to use the mysql80Compatible image, all keyspaces that come after it (in alphabetical order) will incorrectly use the MySQL 8.0 image, even if they are supposed to default to mysql56Compatible. Keyspaces that appear earlier in the alphabetical order are unaffected and continue using the mysql56Compatible image as expected.
Expected Behavior
Keyspaces without keyspace-level image settings should default to the mysql56Compatible image (18.0.7-percona57-hc-cfc6d4900a), regardless of their alphabetical position relative to keyspaces configured with mysql80Compatible.
Observed Behavior
Keyspaces without keyspace-level image settings are incorrectly using the mysql80Compatible image (18.0.7-percona80-hc-635e8e91b7) for the mysqld container.
Steps to Reproduce
Create multiple keyspaces in Vitess Operator v18, ensuring they are sorted alphabetically. Configure one keyspace with the mysql80Compatible image setting. Leave other keyspaces without keyspace-level image settings (defaulting to mysql56Compatible). Observe that keyspaces following the mysql80Compatible keyspace in alphabetical order incorrectly use the MySQL 8.0 image for their mysqld containers.
Examples
Keyspace Here is an example of a VitessKeyspace CR that does not have keyspace-level image settings but is incorrectly using the mysql80Compatible image:
apiVersion: planetscale.com/v2
kind: VitessKeyspace
metadata:
name: xxxxx
creationTimestamp: "2025-06-13T06:38:25Z"
images:
mysqld:
mysql80Compatible: docker.ihs.xxx.net/platform-dbtech/vitess-lite:18.0.7-percona80-hc-635e8e91b7
mysqldExporter: docker-hub.ihs.xxx.net/prom/mysqld-exporter:v0.11.0
vtbackup: docker.ihs.xxx.net/platform-dbtech/vitess-lite:18.0.7-percona57-hc-cfc6d4900a
vtorc: docker.ihs.xxx.net/platform-dbtech/vitess-lite:18.0.7-percona57-hc-cfc6d4900a
vttablet: docker.ihs.xxx.net/platform-dbtech/vitess-lite:18.0.7-percona57-hc-cfc6d4900a
Pod The following pod description highlights the issue. The mysqld container is incorrectly using the mysql80Compatible image (18.0.7-percona80-hc-635e8e91b7) instead of the expected mysql56Compatible image (18.0.7-percona57-hc-cfc6d4900a):
Containers:
mysqld:
Image: docker.ihs.xxx.net/platform-dbtech/vitess-lite:18.0.7-percona80-hc-635e8e91b7
Environment:
MYSQL_FLAVOR: MySQL80
This bug exists in main and we are able to repro this easily.
Define 4 new keyspaces in 101_initial_cluster.yaml, example_a, example_b, example_c & example_d
Override the images: for example_b, set the vttablet: & mysqld.mysql80Compatible: to something different.
Apply and inspect the vitesskeyspaces objects, You will notice that the example_b reflects the different vttablet & mysqld images.
But example_c & example_d also shows the mysqld image change (leaking via the example_b setting). their vttablet image remains intact.
Summary: mysqld: image tag override is being wrongfully applied to rest of the keyspaces in the list.
Issue Reproduction
This error is very easy to reproduce.
Here are the results:
I also conducted an experiment and found that it only affects mysql80Compatible, but not vttablet.
Git Diff
Here is the git diff showing the changes made:
@yydoow @bluecrabs007, thank you for reporting this issue. I have opened a PR to fix it: https://github.com/planetscale/vitess-operator/pull/713.