multiple awx operators in different namespaces with different versions

[philipp1992]

Please confirm the following

• [X] I agree to follow this project's code of conduct.
• [X] I have checked the current issues for duplicates.
• [X] I understand that AWX Operator is open source software provided for free and that I might not receive a timely response.

Feature Summary

Hello, currently the awx-operator does not seem to use different AWX CRD versions for different operator versions. Can you confirm if its supported to install the awx-operator to different namespaces on the same cluster with different versions? E.g. 1.x and 2.x ? Will this cause any issues? Kind regards Philipp

[shanemcd]

Having different versions of the same CRD is not something Kubernetes supports. I think @TheRealHaoLiu had some ideas about how to handle this, will let him chime in here.

[philipp1992]

Having different versions of the same CRD is not something Kubernetes supports. I think @TheRealHaoLiu had some ideas about how to handle this, will let him chime in here.

yes it does but awx-operator just uses the same version for every operator resource https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#specify-multiple-versions

The CustomResourceDefinition API versions field can be used to support multiple versions of custom resources that you have developed. Versions can have different schemas, and conversion webhooks can convert custom resources between versions. Webhook conversions should follow the [Kubernetes API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md) wherever applicable. Specifically, See the [API change documentation](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md) for a set of useful gotchas and suggestions.

[TheRealHaoLiu]

the idea was essentially just use different CRD

having customize template out the name of the CRD itself and change the watches.yml to look for that different CRD

[TheRealHaoLiu]

this side steps the problem of having to deal with CRD versioning altogether

[philipp1992]

the idea was essentially just use different CRD

having customize template out the name of the CRD itself and change the watches.yml to look for that different CRD

any why use such a complicated approach instead of doing it the kubernetes native way? e.g have a new CRD version every time the api changes?

[shanemcd]

@philipp1992 thanks for that information, I didn't realize that. Can you help me understand the flow for upgrading a CR when the version changes?

[philipp1992]

you create a new CRD version every time you change the api (e.g. spec changes) and bundle that specific crd version with your operator

[shanemcd]

I'm trying to understand what steps a user would need to take on their end after we bump CRD version. What steps would be involved to upgrade the AWX object if you created it using an old version of the CRD?

[philipp1992]

a user needs to bump the apiVersion of the awx instance and then the operator needs to handle that (do the upgrade of the deployment etc.)

[shanemcd]

Thank you, that's helpful.

It seems like it could create some confusion as users are accustomed to just upgrading the operator without needing to modify their CR, so we'd need to be careful and think about how this is documented.

I'd also like to develop a better understanding of when to bump the version of a CRD. Does that happen every time you make any kind of change, or only backwards incompatible changes?

[philipp1992]

well its better if the user has to explicitly change the instance. you only need a new CRD version if the interface changes, e.g. you add new spec fields or options

i suggest you to read: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api_changes.md#so-you-want-to-change-the-api

[philipp1992]

or if you dont want the user to handle the CR change, you can use webhooks and mutate them from the operator https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#webhook-conversion

[jpuskar]

The industry standard thing is to use CRD versions. In the CRD's spec it's just:

• v1beta1:
• v1beta2:
• v2beta1:

Super easy to handle.

Operators are usually managed by Helm. Helm can update the CRD's no problem. This is all a normal thing users do and expect.

Totally solves the issue.