Bump mysql container image to one with an aarch64 image

[alexcreasy]

Description

• #266

Bumps the version of the MySQL container image in the overlays/db kustomization to 8.0.39 which is multi-arch (amd64 / aarch64)

How Has This Been Tested?

Deployed to a local kind cluster on an apple silicon MacBook, using official instructions to deploy using kind on macOS.

*https://www.kubeflow.org/docs/components/pipelines/legacy-v1/installation/localcluster-deployment/#kind *https://www.kubeflow.org/docs/components/model-registry/installation/#standalone-installation

The only variation of above was that I deployed to kind using my branch of the manifest: kubectl apply -k "https://github.com/alexcreasy/model-registry/manifests/kustomize/overlays/db?ref=kind"

I've verified I'm able to deploy as above, create a port forward to the service, and query the rest endpoint as indicated in the example above and receive a 200 response.

Merge criteria:

• [x] The commits and have meaningful messages; the author will squash them after approval or will ask to merge with squash.
• [x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
• [x] The developer has manually tested the changes and verified that the changes work

[alexcreasy]

Ahhh... yay the old "it worked on my laptop" scenario 🤦

I guess the new image isn't being loaded into kind on the CI jobs.

[rareddy]

isn't MR using MariaDB not mySQL that too a version that is matched with KFP?

[tarilabs]

I believe we had to stick with the currently used version, as that's the one tested to be working with MLMD. Further, this won't solve the issue you linked, for the reasons I've summarized in https://github.com/kubeflow/model-registry/issues/266#issuecomment-2287945898

Let me know your thoughts :)

[alexcreasy]

@tarilabs thanks for taking a look, I believe it will solve my issues, as I'm trying to deploy model registry without kubeflow (full explanation of what I'm trying to do on the ticket) - granted I've only run a few curl requests against the MR API on my local kind deployment, so there could be some hidden issue I'm not aware of, but so far they work as expected using the modified manifests on my fork.

Let me know if I've misunderstood the core problem you described :)

[rareddy]

I believe we had to stick with the currently used version, as that's the one tested to be working with MLMD.

I was under impression that KFP team use MariaDB for writing some windowing functions over it which were only supported in MariaDB and working fine for them. May be this only used for Red Hat's distribution.

[tarilabs]

I want to augment the comment in https://github.com/kubeflow/model-registry/pull/267#pullrequestreview-2244881693 with the information that the default vanilla installation uses a custom image of mysql from google container registry:

https://github.com/kubeflow/manifests/blob/a38c2be88fbafb0844c0231f0062e4b3719d4737/apps/pipeline/upstream/third-party/mysql/base/mysql-deployment.yaml#L51

👉 gcr.io/ml-pipeline/mysql:8.0.26

[tarilabs]

I still wonder why it fails at

kubectl wait --for=condition=available -n kubeflow deployment/model-registry-db --timeout=5m

[alexcreasy]

@tarilabs I'm not sure, I haven't looked into it yet - it's strange, I can deploy this to kind locally. I'll dig into this next week and see what I can find.

[alexcreasy]

I wonder if it's some issue with pulling the image? At a guess (and this is a complete guess) I wonder if the db image needs to be manually loaded into kind for the deployment to be able to use it.

[tarilabs]

I wonder if it's some issue with pulling the image? At a guess (and this is a complete guess) I wonder if the db image needs to be manually loaded into kind for the deployment to be able to use it.

Maybe, but this looks not being a problem for mysql 8.0.3 🤔

[tarilabs]

let's see if 7a8bded is of any help in digging further into the actual error 🧐

[tarilabs]

let's see if 7a8bded is of any help in digging further into the actual error 🧐

with the log extraction, locate the following:

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

which brings up:

• https://github.com/kubeflow/manifests/issues/2507
• https://github.com/kubeflow/manifests/issues/2508
• https://kind.sigs.k8s.io/docs/user/known-issues/#apparmor
• https://github.com/actions/runner-images/issues/181#issuecomment-610874237

so the solution appears to be removing the AppArmor profile for mysql in the GHA runner...

[tarilabs]

All green now, awesome 🚀

Now, if only we had some answer on...

• https://github.com/kubeflow/pipelines/discussions/11224

[tarilabs]

btw, I assume this resolves #266

[tarilabs]

btw, this also should be of great advantage to https://github.com/kubeflow/model-registry/issues/327#issuecomment-2324155752

[tarilabs]

@alexcreasy with the latest rebase event from https://github.com/kubeflow/model-registry/pull/267#event-15955837654 I assume we lost commits:

• 7ad916e
• bd3563d
• 7c4f26c

that were discussed with https://github.com/kubeflow/model-registry/pull/267#issuecomment-2494530746

I've rebased again on main and pushed here. Kindly account as you will use your local branch with this PR open.

Hope this helps!

[alexcreasy]

@tarilabs ahh thanks, I'll test this locally now, do you need me to do anything else?

[tarilabs]

as discussed in today's KF MR biweekly meeting, proceeding to merge.

[google-oss-prow[bot]]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tarilabs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• ~~OWNERS~~ [tarilabs]

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment