Installation fails on Kubernetes v1.24

[maxlemieux]

Description

Installation fails on Kubernetes v1.24

Go Version

go version go1.20.4 darwin/amd64

Current behavior

Installation following the readme fails on Kubernetes v1.24.

Errors when applying the kustomize build (both with latest Kustomize, or with v3.8.10) - it uses deprecated API versions for several resources.

Expected behavior

Installation succeeds on recent Kubernetes versions

Steps To Reproduce

Steps to reproduce the behavior:

1. Have a Kubernetes 1.24 cluster (probably affects several earlier versions since the API changes)
2. Install cert-manager and test it successfully according to cert-manager docs (the command in the readme failed, but the Helm chart from cert-manager installed fine)
3. Install newrelic-kubernetes-operator with kustomize
4. See error

Debug Output (if applicable)

~/kustomize build https://github.com/newrelic/newrelic-kubernetes-operator/config/default | kubectl apply -f -
+ kubectl apply -f -
namespace/newrelic-kubernetes-operator-system unchanged
role.rbac.authorization.k8s.io/newrelic-kubernetes-operator-leader-election-role unchanged
clusterrole.rbac.authorization.k8s.io/newrelic-kubernetes-operator-manager-role configured
clusterrole.rbac.authorization.k8s.io/newrelic-kubernetes-operator-proxy-role unchanged
clusterrole.rbac.authorization.k8s.io/newrelic-kubernetes-operator-secret-reader unchanged
rolebinding.rbac.authorization.k8s.io/newrelic-kubernetes-operator-leader-election-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/newrelic-kubernetes-operator-manager-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/newrelic-kubernetes-operator-proxy-rolebinding unchanged
clusterrolebinding.rbac.authorization.k8s.io/newrelic-kubernetes-operator-secrets-rolebinding unchanged
service/newrelic-kubernetes-operator-controller-manager-metrics-service unchanged
service/newrelic-kubernetes-operator-webhook-service unchanged
deployment.apps/newrelic-kubernetes-operator-controller-manager unchanged
resource mapping not found for name: "alertsapmconditions.nr.k8s.newrelic.com" namespace: "" from "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "alertschannels.nr.k8s.newrelic.com" namespace: "" from "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "alertsnrqlconditions.nr.k8s.newrelic.com" namespace: "" from "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "alertspolicies.nr.k8s.newrelic.com" namespace: "" from "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "apmalertconditions.nr.k8s.newrelic.com" namespace: "" from "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "nrqlalertconditions.nr.k8s.newrelic.com" namespace: "" from "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "policies.nr.k8s.newrelic.com" namespace: "" from "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "newrelic-kubernetes-operator-serving-cert" namespace: "newrelic-kubernetes-operator-system" from "STDIN": no matches for kind "Certificate" in version "cert-manager.io/v1alpha2"
ensure CRDs are installed first
resource mapping not found for name: "newrelic-kubernetes-operator-selfsigned-issuer" namespace: "newrelic-kubernetes-operator-system" from "STDIN": no matches for kind "Issuer" in version "cert-manager.io/v1alpha2"
ensure CRDs are installed first
resource mapping not found for name: "newrelic-kubernetes-operator-mutating-webhook-configuration" namespace: "" from "STDIN": no matches for kind "MutatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"
ensure CRDs are installed first
resource mapping not found for name: "newrelic-kubernetes-operator-validating-webhook-configuration" namespace: "" from "STDIN": no matches for kind "ValidatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"
ensure CRDs are installed first

Additional Context

Updating these references to the current v1 APIs does not solve this, as there are other errors reported by kustomize with recent kustomize versions. Running kustomize edit fix does not automatically fix those errors.

I tested with a version of kustomize, 3.8.10, from 2021. This resolved the kustomize errors seen with the new version but showed a different error:

Error: no matches for IdId admissionregistration.k8s.io_v1_MutatingWebhookConfiguration|~X|mutating-webhook-configuration; failed to find unique target for patch admissionregistration.k8s.io_v1_MutatingWebhookConfiguration|mutating-webhook-configuration
error: no objects passed to apply

So there seems to be something else needed.

[TashikMoin]

Still no update on this?

[bra-fsn]

Same here. If this is an abandoned project, it would be worth noting it in the README.

[woehrl01]

I just invested some time to fix the apply via kustomize: https://github.com/woehrl01/newrelic-kubernetes-operator/pull/1/files

The problem is now that all the go modules are outdated so that the webhook validation fails with:

Error from server (InternalError): error when creating "./newrelic-op.yaml": Internal error occurred: failed calling webhook "malertspolicy.kb.io": received invalid webhook response: expected response.uid="06efc0e3-2c46-404b-94bc-0adfc97d117e", got ""

I am uncertain if it's worth the trouble to update all those dependencies, too.

One question for the maintainers of this library, will a PR be reviewed and accepted? @thande

[kalvkusk]

@thande I migrated the CRDs to work with current v1 api extensions.

https://github.com/newrelic/newrelic-kubernetes-operator/pull/152

[kalvkusk]

@woehrl01 Checkout my PR, I fixed the admission hooks. Tested, it works ;)