E2E test for kfctl upgrade

[jlewi]

trafficstars

/kind feature

Creating a separate issue to track adding an E2E test for kfctl upgradability.

kubeflow/kubeflow#3727 is the uber tracking issue for supporting upgrades.

/cc @richardsliu /assign @jlewi

[jlewi]

Here's my initial thought.

An E2E test probably looks like the folling

1. Build kfctl
2. Deploy v0X
3. Test Kubeflow is ready
4. Upgrade Kubeflow
5. Verify Kubeflow is ready

So we will need to make the following changes.

• Create a new py_func E2E test based on our existing ksonnet tests
• Split kfctl_go_test.py into two steps
  • Build the kfctl binary
  • Deploy kubeflow

I will probably start by creating a minimal py_func that probably just builds and deploys kubeflow.

[jlewi]

Update:

• I have a draft PR #4148 to port the kfctl E2E test workflow to python.
• Once that PR is submitted I can work on using that as the basis of an E2E test for kfctl upgrade.
• I estimate at least another 2 weeks to get everything done.

[PatrickXYS]

I think kfctl/upgrade is not supported at this moment, but will be an action item on roadmap

/assign