Kubernetes OpenAPI v3

[apelisse]

Hi there,

Kubernetes (and my team specifically) has been trying to migrate Kubernetes to use OpenAPI v3 and the feature is now in beta in Kubernetes v1.24. This means that both built-in types and CRDs are described through the same endpoint, with new improved caching mechanisms. I'm curious if this project has any interest into validating resources using the newly served OpenAPI rather than the existing v2 OpenAPI and its huge limitations to CRD validation.

Thanks!

[yannh]

Hi @apelisse , maintainer of Kubeconform here - I would be very interested. Kubeval and Kubeconform do not validate directly against openapi, but convert the file to json schemas first https://github.com/yannh/kubernetes-json-schema

If you know of a good Go library to use to validate against OpenAPI v3, and Kubernetes provided an OpenAPI v3 schema, I would most likely upgrade Kubeconform to use that.

[apelisse]

Absolutely, we're using https://github.com/getkin/kin-openapi internally to validate that our OpenAPI definition is valid, but you can also validate data against an OpenAPI definition! (I think). Let me know how this goes, happy to work with you toward that goal!

[yannh]

@apelisse to clarify - you are talking about using the OpenAPIs as served by the Kubernetes endpoint? The way I use Kubeval/Kubeconform (and I think as do many others do) is to validate manifests within a CI, without direct access to a Kubernetes cluster. Currently they both use a separate repo for validation https://github.com/yannh/kubernetes-json-schema , derived from the swagger file, and it would be great to be able to retire this conversion.

What I was wondering about was using kin-openapi to validate directly against https://github.com/kubernetes/kubernetes/tree/release-1.24/api/openapi-spec/v3 ? I'm looking into this but I admit I'm not exactly an expert on the differences between openapi v2/v3/swagger/ and json schemas :grimacing:

[apelisse]

For general lack of access, I'm planning on working on a tool to extract it from clusters and push it either as a file, base-layer docker image, but indeed you can get it from https://github.com/kubernetes/kubernetes/tree/release-1.24/api/openapi-spec/v3 too, though it won't include your custom-resource types, which is frustrating (we definitely want to validate custom-resources).

I think that kin-openapi example shows an example though it validates the routes which is probably not what we want. Also there are definitely kubernetes specific things that we want to validate (especially regarding associative lists) so it's not going to be that obvious.

[yannh]

For CRDs - I usually put them all in a local folder to make them available (see https://github.com/yannh/kubeconform/#overriding-schemas-location---crd-and-openshift-support )

If you ever managed to come with an example how to validate a single manifest against the openapiv3 with kin-openapi I could probably update Kubeconform, but I've been looking at https://pkg.go.dev/github.com/getkin/kin-openapi/openapi3?utm_source=godoc scratching my head how to do that in the past :grimacing: