Support target cluster version

[jpreese]

If you have a bundle of manifests, it would be nice to be able to verify that manifest against a specific version of Kubernetes.

For example, if you have a bundle.yaml that you intended to deploy to Kubernetes v1.17.0, it would be desirable to programmatically choose with set of Deprek8ion policies to run. In this case you'd include 1.16 and 1.17, but leave the others out.

Maybe a folder per Kubernetes version? e.g. 1.17/kubernetes-1.16.rego 1.17/kubernetes-1.17.rego and then conftest test bundle.yaml -p deprek8ion/$KUBERNETES_VERSION

[swade1987]

Hi @jpreese I’ve been thinking about the best way to structure this for a while now.

So you’re saying that policies under folders which are specific to that Kubernetes version?

The issue is what to do when you want to test all versions, is there an easy way to do this?

[jpreese]

As a POC in our repo I just did this:

1.16/kubernetes-1.16.rego

1.17/kubernetes-1.16.rego 1.17/kubernetes-1.17.rego

...

all/

That way when you do Conftest -p 1.17, every version below it gets tested as well.

It's a lot of duplication, but given how small the project is, it solved the problem without much effort.

Though I'm open to a cleaner solution, especially one in the upstream so we can just continue to use Conftest pull

[ckotzbauer]

Hi, I built a tool which uses the policies from this repo with a version-check. I detect the minor version of the filename, so no folder is required...

But I think this would not work correctly right now, as I mentonied here: #12

I saw, that some deprecations were present there [1.19] (e.g. admissionregistration.k8s.io/v1beta1). But those are deprecated since 1.16 and not since 1.19 (which is said in the warn-message) and are no longer served from 1.19. Maybe they should be moved to the 1.16 file? I didn't check other files yet...

Edit: But you are right, without an external tool it is not possible right now, since conftest doesn't respect filenames...

[swade1987]

I am wondering if I just release a container per kubernetes minor release containing just the deprecations for that release, thoughts?

[swade1987]

@ckotzbauer @jpreese would you guys prefer a container per minor release or a directory structure similar to what @jpreese suggests? Waiting for feedback before implementation to make sure we are all aligned.

[jpreese]

If I'm understanding correctly, I don't think you could do just the deprecations for that release. If I have a bundle containing some Kubernetes manifests, and I want to deploy it to Kubernetes 1.17, you'd need to validate that bundle against both 1.16 and 1.17 yeah?

The problem I was running into was that just looking at a single version.rego, it was missing other manifests because they were deprecated in a previous release.

[swade1987]

So if I re-structured the repo for a directory per minor release (e.g. 1.17) and then inside that directory have all the deprecations for 1.17 and earlier, would that work?

[jpreese]

That's how I currently have it in our repository and it seems to be working.

Technically, when you generate the .rego you could just dump the previous versions in the file as well (i.e. kubernetes-1.17.rego would contain the rules for both 1.16 and 1.17 in a single file). It's a bit more concise anyway.

[swade1987]

I was actually wondering if having kubernetes-1.16-deny.rego and kubernetes-1.16-warn.rego makes sense.

That way you have a location for all API versions that are denied by the specific version of Kubernetes in one file.

Then in the other file, you have the advanced warnings, thoughts?

[jpreese]

I dont think we'd personally use it. Our intent is just to:

• Conftest pull
• Conftest test bundle.yaml -p some_kubernetes_version
• Go forth and fix all the things

Not strongly opinionated on how warn/deny rules are split up, though given that there's no real good way to control warnings/errors in Conftest, maybe some teams would see value in splitting it up.

[swade1987]

I’ll get a new branch and PR setup tomorrow at some point and add you as a reviewer 😀

[ckotzbauer]

That sounds, good. Yes, target-version should cover this and previous versions. A splitting of warnings and errors is not necessary for me. If this is built with a directory structure or separated docker-images doesn't matter.

[swade1987]

@ckotzbauer @jpreese I am actually thinking that specific containers for target versions may be easier.

[jpreese]

Given that these policies shouldn't change once created, we'll most likely stick with the folder structure approach. Having to run a docker container to lint the manifests for every PR seems a bit heavy-handed.

That said, I do think if you were to make a container for each version and bundled it with conftest, people would use it.

[swade1987]

I have just gone back and forward with the directory structure.

My issue is the docker container itself, is it useful or not really?