pipelines icon indicating copy to clipboard operation
pipelines copied to clipboard

[WIP]chore(backend): migrate GORM v1 to v2

Open kaikaila opened this issue 7 months ago • 7 comments

Summary

This PR migrates the entire ORM layer of the Kubeflow Pipelines backend from GORM v1 (github.com/jinzhu/gorm) to GORM v2 (gorm.io/gorm), covering:

  • ✅ All core model definitions in backend/src/apiserver/model
  • ✅ Full migration of the Execution Cache system under backend/src/cache/...

Migration Details

Models Migration (GORM v1 → v2)

  • Replaced all gorm:"primary_key" with gorm:"primaryKey"
  • Ensured sql.Null* fields are handled consistently (where applicable)
  • Switched from deprecated NewRecord() to .Create().Error-based flow
  • Updated all AutoMigrate() calls to conform to GORM v2 APIs
  • Removed all legacy GORM v1 imports

Execution Cache Migration

  • Updated NewFakeDB, ClientManager, and associated test helpers to GORM v2
  • Migrated store logic in execution_cache_store.go to use db.Create(...).Scan(...) patterns
  • Removed deprecated NewRecord() usage and added structured error wrapping
  • Restored non-unique index on ExecutionCacheKey to match v1 behavior (tests rely on repeated keys + latest record retrieval)

Client Manager Migration

Changes • Remove manual index creation block in InitDBClient (client_manager.go). • Add type:varchar(191) + index:...,priority:... tags in: • model.Run.ExperimentId and model.Run.Namespace • model.RunDetails.Conditions and model.RunDetails.FinishedAtInSec • model.RunDetails.CreatedAtInSec • Add uniqueIndex:name_namespace_index,priority:... tags in model.Pipeline.Name and model.Pipeline.Namespace • Retain original index names for backward compatibility and least-surprise.


Notes

This PR supersedes any previous model-level-only migration PRs (e.g., chore/orm-migrate-resource-reference) by consolidating all ORM transitions into a unified update.
It also ensures that GORM v1 is no longer referenced anywhere in the backend codebase.


Checklist

  • [x] All tests pass locally (go test ./backend/...)
  • [x] go.mod and imports updated to use GORM v2
  • [x] No GORM v1 usage remains
  • [x] Migration is backward-compatible with existing data schema where possible

kaikaila avatar May 25 '25 06:05 kaikaila

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign james-jwu for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

google-oss-prow[bot] avatar May 25 '25 06:05 google-oss-prow[bot]

Hi @kaikaila. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

google-oss-prow[bot] avatar May 25 '25 06:05 google-oss-prow[bot]

🚫 This command cannot be processed. Only organization members or owners can use the commands.

github-actions[bot] avatar May 25 '25 07:05 github-actions[bot]

@kaikaila: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

google-oss-prow[bot] avatar May 26 '25 07:05 google-oss-prow[bot]

🚫 This command cannot be processed. Only organization members or owners can use the commands.

github-actions[bot] avatar May 26 '25 07:05 github-actions[bot]

/ok-to-test

zazulam avatar May 26 '25 12:05 zazulam

/retest

kaikaila avatar Jun 17 '25 21:06 kaikaila

Hi @HumairAK I wanna bring this to your attention. In backend/src/apiserver/model/task.go line 26, the field name RunId ≠ column name RunUUID. This may cause confusion when using foreignKey: in GORM v2. Happy to refactor if we want to align them.

kaikaila avatar Jun 24 '25 07:06 kaikaila

Closed in favor of #12013 with a more accurate branch name and scope. Please continue reviewing there!

kaikaila avatar Jun 25 '25 07:06 kaikaila