Refactoring common operator tests

[richardsliu]

There are common Python test code in: https://github.com/kubeflow/pytorch-operator/tree/master/test https://github.com/kubeflow/tf-operator/tree/master/test

Consider how these can be refactored to reduce duplication.

[richardsliu]

/cc @johnugeorge /cc @terrytangyuan /cc @jian-he

[gaocegege]

Could we add more unit test in common operator? I think it is hard to import integration test or system test since the operator is not a real k8s operator.

[merlintang]

I would be better that we can refactor the test case, and provide the dev document for running test.

[gaocegege]

@richardsliu Can we make TestJob a real CRD, then mock the kube-apiserver to run tests in job_controller?

The code coverage is too low. If we do not have a real CRD, we cannot test some logic of the common operator.

[richardsliu]

That is the plan. We can add e2e tests with the TestJob as well.