Multi-Version CRs support

[a-hilaly]

Context

All ACK controllers start out with an Alpha API version (v1alpha1) that evolves from release to release, most likely due to updates in aws/aws-sdk-go OpenAPI Schemas or because of changes in the generator.yaml (ignored fields, renamed fields, field made secrets etc...) Eventually most of these APIs will need to move to a more stable API ( v1beta1, v1beta2... v1...) and restricted from making breaking changes to it.

However to simplify ACK users experience when dealing with controllers/CRDs upgrades we need to keep support for multiple API versions (if not all of them) and gradually deprecate old versions when needed (see kubernetes deprecation policy)

Implementation details:

Changes to ack/code-generator

• [x] ack-generate should be able to generate API definitions for different ApiVersions (currently supported with --api-version flag)
• [x] ack-generate apis should save a copy of generator.yaml within the ApiVersions directory (apis/v*/)
• [x] ack-generate should save the parameters used to generate a specific ApiVersion (aws-sdk-go, ack--generate versions, ENV Vars, etc ...) see #809
• [x] controller flags should allow to enable webhook server and set it's host and port
• [x] runtime should contain a package to systematize webhook registration and listing
• [x] ack-generate controller should generate code that registers conversion webhooks on controller startup (probably done in main.go) https://github.com/aws-controllers-k8s/code-generator/pull/111
• [x] ack-generate should start using gogit to load/checkout aws-sdk-go specific versions https://github.com/aws-controllers-k8s/code-generator/pull/108
• [ ] ack-generate controller should generate conversion functions by loading all aws-sdk-go version schemas and their related generator.yaml and computing the CRD "type diffs"
• [ ] Change the scripts/build-controller.sh to call controller-gen with specific arguments when dealing with multiple versions
• [ ] Change the scripts/release-controller.sh to include the new CRD Schemas and optionally create a CA (Certificate Authority) service

Documentation:

• [ ] Document how code generation works for multiple version works (conversion functions, webhooks, Auth/CA service)
• [ ] Document how does a controller support two or more API versions (data flow description + diagram)
• [ ] Document the required steps from K8s cluster administrators to setup multi-version ACK controllers

/cc @aws-controllers-k8s/code-generator-maintainer @aws-controllers-k8s/runtime-maintainer @aws-controllers-k8s/community-maintainers

[RedbackThomson]

we need to keep support for multiple API versions

Would you mind clarifying what the Kubernetes definition of "support for" means in this context? Is it simply that we are able to convert to/from them, or are there stronger definitions about their usage?

ack-generate should start using gogit to load/checkout aws-sdk-go specific versions

Is this related to versioning, or just an aside?

create a CA (Certificate Authority) service

What purpose does a CA serve for CRDs?

Is there also some way we can ensure we do not modify a previous version of a released CRD? That is, if we've released v1alpha2 we must somehow lock v1alpha1 to a given version.

[a-hilaly]

Would you mind clarifying what the Kubernetes definition of "support for" means in this context? Is it simply that we are able to convert to/from them, or are there stronger definitions about their usage?

Yes, simply that we will be able to convert to/from them, by always regenerating conversion functions whenever there is a CR version bump (latest version is always the "Hub" version)

Is this related to versioning, or just an aside?

Not directly related to versioning. It's gonna simplify the way we load AWS API Specifications from aws-sdk-go

What purpose does a CA serve for CRDs?

The CA will issue certificates to ensure secure communication between the kube-apiserver and a given ACK controller. The webhook server will be running within the service controller (https://github.com/kubernetes-sigs/controller-runtime/blob/master/alias.go#L103-L104)

Is there also some way we can ensure we do not modify a previous version of a released CRD?

We can do that by saving an output file containing the files checksum whenever we regenerate an API version. However conversion functions will always change whenever we update/create a new API version - so maybe we can just exclude the conversion functions when computing the checksum ?

[ack-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle stale

[a-hilaly]

/remove-lifecycle stale

[muvaf]

@A-Hilaly If I understand correctly, the apiVersion is specified per-group, right? Not per-CRD? In Crossplane, we go for per-CRD to comply with Kubernetes and also CRDs in the same group usually don't have the same maturity level. Do you see that case arising in ACK as well or do you consider the whole group as a unit that will be versioned together?

[ack-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with /close. Provide feedback via https://github.com/aws-controllers-k8s/community. /lifecycle stale

[vijtrip2]

/lifecycle frozen