PROPOSAL - create a "pool" of cloud credits for Kubeflow testing

[thesuperzapper]

This proposal is pretty simple, the general idea is that we have use-cases that require us to spin up full Kubeflow clusters and perform manual testing.

Logistically, I think the easiest way to do this is a "pool" of cloud credits which can be accessed by some designated list of people within the Kubeflow community for the purposes of testing Kubeflow.

I would love to hear from the main cloud providers about if this is feasible:

• @kubeflow/aws
• @kubeflow/azure
• @kubeflow/google

---

This was raised by @js-ts in the latest community meeting, where he shared his struggles in improving our kubeflow/examples repo without a remote Kubernetes cluster to run Kubernetes on (with GPUs, for example).

[terrytangyuan]

We already have test infrastructure on AWS: https://github.com/kubeflow/testing/tree/master/aws

[js-ts]

Thanks @thesuperzapper for raising this issue , Earlier there was Kubeflow Auto Deploy Infrastructure It provided option to run both manual and automated tests

The primary purpose of the auto deployed clusters is to provide up to date clusters for manual and automatic tests. For example, test in kubeflow/examples should be configured to select an auto deployed cluster and then run against that.

since the option has been discontinued the no of contributors and contributions have gone down significantly

I've taken the initiative to remove outdated examples and add new ones even update some examples , writing automated tests for every example is not feasible, for that we need manual option to test things , I created a example to contribute to the examples repo and many more are will be coming from a diverse group of contributors , for testing the example a manual deployment mode is necessary , even though I've tested it locally , I think it should be tested to check whether if it does run into an infinite loop or not and other things , also for other examples which use a cloud specific service and resources (GPU,TPU) its hard to test it locally , an option to manually test examples is the only way I think can solve this issue for that an alternative to AUTODEPLOY or as @thesuperzapper suggested providing a "pool" of cloud credits should be provided which can be accessed by some designated list of people within the Kubeflow community for the purposes of testing Kubeflow.

In Kubeflow community survey tutorials was the second biggest gap in in Kubeflow after documentation ,providing resources, contributing new examples, creating issues can fill this GAP

also the kubeflow/testing repo uses examples from the kubeflow/examples , I've not started deleting/removing examples or moving them to another branch as I think doing that might cause problems in testing wdyt @kubeflow/oncall-testing or maintainers of kubeflow/testing

@kubeflow/arrikto @kubeflow/aws @kubeflow/azure @kubeflow/cisco @kubeflow/google @kubeflow/ibm @kubeflow/red-hat It would be great if you could provide resources or credits as @thesuperzapper mentioned for testing/updating/adding examples that are vendor specific. if any other WG/repo maintainers needs this please mention it below, thanks everyone

[thesuperzapper]

We already have test infrastructure on AWS: https://github.com/kubeflow/testing/tree/master/aws

@terrytangyuan is that infrastructure available for "manual testing" or only "automated testing"?

[terrytangyuan]

They can be triggered by PR comments. If that's considered "manual", then yes it's available for both.

[js-ts]

@terrytangyuan can you test this example using https://github.com/kubeflow/testing/tree/master/aws

[terrytangyuan]

@terrytangyuan can you test this example using https://github.com/kubeflow/testing/tree/master/aws

Please check out the relevant documentation and raise an issue if you have questions in kubeflow/testing.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.