Operator projects using the removed APIs in k8s 1.22 requires changes.

[camilamacedo86]

trafficstars

Problem Description

Kubernetes has been deprecating API(s), which will be removed and are no longer available in 1.22. Operators projects using these APIs versions will not work on Kubernetes 1.22 or any cluster vendor using this Kubernetes version(1.22), such as OpenShift 4.9+. Following the APIs that are most likely your projects to be affected by:

• apiextensions.k8s.io/v1beta1: (Used for CRDs and available since v1.16)
• rbac.authorization.k8s.io/v1beta1: (Used for RBAC/rules and available since v1.8)
• admissionregistration.k8s.io/v1beta1 (Used for Webhooks and available since v1.16)

Therefore, looks like this project distributes solutions via the Red Hat Connect with the package name as kube-arangodb and does not contain any version compatible with k8s 1.22/OCP 4.9. Following some findings by checking the distributions published:

• kube-arangodb.v1.0.1: this distribution is using APIs which were deprecated and removed in v1.22. More info: https://kubernetes.io/docs/reference/using-api/deprecation-guide/#v1-22. Migrate the API(s) for CRD: (["arangobackuppolicies.backup.arangodb.com" "arangobackups.backup.arangodb.com" "arangodeploymentreplications.replication.database.arangodb.com" "arangodeployments.database.arangodb.com"])

NOTE: The above findings are only about the manifests shipped inside of the distribution. It is not checking the codebase.

How to solve

It would be very nice to see new distributions of this project that are no longer using these APIs and so they can work on Kubernetes 1.22 and newer and published in the Red Hat Connect collection. OpenShift 4.9, for example, will not ship operators anymore that do still use v1beta1 extension APIs.

Due to the number of options available to build Operators, it is hard to provide direct guidance on updating your operator to support Kubernetes 1.22. Recent versions of the OperatorSDK greater than 1.0.0 and Kubebuilder greater than 3.0.0 scaffold your project with the latest versions of these APIs (all that is generated by tools only). See the guides to upgrade your projects with OperatorSDK Golang, Ansible, Helm or the Kubebuilder one. For APIs other than the ones mentioned above, you will have to check your code for usage of removed API versions and upgrade to newer APIs. The details of this depend on your codebase.

If this projects only need to migrate the API for CRDs and it was built with OperatorSDK versions lower than 1.0.0 then, you maybe able to solve it with an OperatorSDK version >= v0.18.x < 1.0.0:

$ operator-sdk generate crds --crd-version=v1 INFO[0000] Running CRD generator.
INFO[0000] CRD generation complete.

Alternatively, you can try to upgrade your manifests with controller-gen (version >= v0.4.1) :

If this project does not use Webhooks:

$ controller-gen crd:trivialVersions=true,preserveUnknownFields=false rbac:roleName=manager-role paths="./..."

If this project is using Webhooks:

1. Add the markers sideEffects and admissionReviewVersions to your webhook (Example with sideEffects=None and admissionReviewVersions={v1,v1beta1}: memcached-operator/api/v1alpha1/memcached_webhook.go):

2. Run the command:

$ controller-gen crd:trivialVersions=true,preserveUnknownFields=false rbac:roleName=manager-role webhook paths="./..."

For further info and tips see the blog.

Thank you for your attention.

[ajanikow]

Hello!

All API versions are already updated in latest Operator (1.2.3). Next API version update will be for Kubernetes 1.25+ (only warning we see currently during tests on Kubernetes 1.21+)

Best Ragards, Adam

[camilamacedo86]

Hi, would we have any update on this?

See that we are very close to the release data and fix the projects seems not very hard. See how to fix it in the first comment. Then, would be great to be able to check a new version of your project distributed which is compatible with 4.9.

[camilamacedo86]

HI @ajanikow,

The solution published in RedHat connect (kube-arangodb.v1.0.1) still using the removed APIs at least for CRD and does not work on 4.9 at all. Could you please ensure that you publish a workable solution on 4.9? Note that if you do not be able to do that asap your package will be removed from 4.9 and after that, your first version workable on 4.9 cannot use replaces. That means the easy way here is to ensure a new version is published there.

c/c @informalict

[ajanikow]

Hello @camilamacedo86!

Manifest creation changed on Marketplace. Itis easy to publish new image (already done), but to be able to update it in Operator Solution in RedHat Connect we have to change manifests. We are working currently on this part.

Best Regards, Adam.

[thorroed]

Latest Operator (1.2.3) still references apiextensions.k8s.io/v1beta1 in arango-storage.yaml and in its template. Following change seems to fix it (for me for now).

--- arango-storage.yaml.1       2021-10-13 19:24:01.615716880 -0400
+++ arango-storage.yaml 2021-10-13 19:21:50.358766355 -0400
@@ -13,7 +13,7 @@
       release: storage
 ---
 # Source: kube-arangodb/templates/storage-operator/crd.yaml
-apiVersion: apiextensions.k8s.io/v1beta1
+apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
     name: arangolocalstorages.storage.arangodb.com
@@ -33,7 +33,22 @@
             - arangostorage
         singular: arangolocalstorage
     scope: Cluster
-    version: v1alpha
+    versions:
+        - name: v1alpha
+          served: true
+          storage: false
+          schema:
+            openAPIV3Schema:
+              type: object
+              x-kubernetes-preserve-unknown-fields: true
+        - name: v1
+          served: true
+          storage: true
+          schema:
+            openAPIV3Schema:
+              type: object
+              x-kubernetes-preserve-unknown-fields: true
+
 ---
 # Source: kube-arangodb/templates/storage-operator/cluster-role.yaml
 apiVersion: rbac.authorization.k8s.io/v1

[flickerfly]

I'd love to see this solved as I'm real close to running into this being a problem.

Server Version: version.Info{Major:"1", Minor:"21"

[ajanikow]

@flickerfly it is mostly solved in 1.2.8, in 1.3.0 full fix for Storage CRD will be in place.