Integration tests

[felixfbecker]

Bugs like #51 could be prevented by having tests that run against an actual Kubernetes cluster. This would catch wrong assumptions like return types of certain endpoints.

It's a fair amount of work to get right, but I actually already have this set up for PSKubectl. It runs minikube in Travis, kubectl apply's test resources, runs commands and then does assertions against output of kubectl -o json. Tests are written in PowerShell with Pester which makes interacting with kubectl very easy.

Alternatively we could just consider PSKubectl an external test suite of this, maybe even trigger a build on every published version from a branch. Or CI here could run the test suite of PSKubectl by cloning the GitHub repo.

[tintoy]

I love this idea - hadn’t had time to do it before but I agree it’s totally worth doing :)

I’ll take a look at PSKubectl’s scripts to see what it would take to set this up (external test suite is a good option, I think, as I don’t expect it to be quick to run?)

[felixfbecker]

It's fairly quick actually. Booting up Minikube takes a bit (45s), but every test is faster than you'd think. It kubectl apply's the resources before every test and uses e.g. a deployment of a hello world docker image with imagePullPolicy: IfNotPresent, so recreating pods takes about a second before every test.

[tintoy]

Had a quick look over your CI scripts - that all looks pretty achievable, especially since you’ve done most of the hard work already :)

[tintoy]

I guess the only tricky bit would be cleaning up after each test :)

Maybe a decorator for IXXXClientVX that captures names of resources created so we can run kubectl delete after the test is complete?

[felixfbecker]

What do you need to cleanup? The next test will just kubectl apply the resources again so they get reset

[felixfbecker]

kubectl apply --prune also deletes resources

[tintoy]

What about tests that use the client to create resources?

I guess I could just write a script to delete all resources in a given namespace or something...

[tintoy]

For gets or deletes though, yeah, it’d be fine.

[felixfbecker]

As said kubectl apply --prune should delete all resources that are not present in the input YAML afaik

[tintoy]

Oh - interesting, I hadn’t heard of that usage but that would do the trick :)

[tintoy]

Ah, apparently that only removes resources created by kubectl apply (not ones created by the client). Still, sounds useful :)

[felixfbecker]

kubectl delete --all deletes all resources in a namespace :)