Unable to install scylla-operator on EKS with custom CNI adapter

[parakr]

Describe the bug Unable to run operator on EKS with custom CNI adapter

To Reproduce Steps to reproduce the behavior:

1. Install operator via helm helm install scylla-manager scylla/scylla-manager --create-namespace --namespace scylla-manager
2. See error Internal error occurred: failed calling webhook "webhook.scylla.scylladb.com": Post "https://scylla-operator-webhook.scylla-operator.svc:443/validate?timeout=10s": Address is not allowed

Expected behavior Operator should be running.

Environment:

• Platform: EKS with custom CNI adapter

Additional context Webhook does not work when custom CNI adapter is used. Helm chart should have the ability to override hostNetwork value for webhook-server Deployment.

[tnozicka]

I wonder whether such cluster passes the Kubernetes conformance tests. Looking at a random webhook test, say https://github.com/kubernetes/kubernetes/blob/4e0069b9097ea19bb5af642893175dd092e64920/test/e2e/apimachinery/webhook.go#L403-L410 and the definition for the deployment https://github.com/kubernetes/kubernetes/blob/4e0069b9097ea19bb5af642893175dd092e64920/test/e2e/apimachinery/webhook.go#L832-L836 would suggest it fails. If that would be the case that would no longer be a "Kubernetes" cluster.

That said, I'd not mind a PR exposing the option but the real issue seems to be with the cluster.

[parakr]

Honestly, I'm not sure if this configuration passes conformance test, but custom CNI adapter is common way how to bypass at least AWS EKS limitations (number of pods per node). Cert-manager which is recommended tool for self signed certificates on your site also has this issue and it already contains option to set hostNetwork to true. https://cert-manager.io/v1.3-docs/installation/compatibility/ Thank you

On Wed, Mar 30, 2022 at 9:24 AM Tomáš Nožička @.***> wrote:

I wonder whether such cluster passes the Kubernetes conformance tests. Looking at a random webhook test, say

https://github.com/kubernetes/kubernetes/blob/4e0069b9097ea19bb5af642893175dd092e64920/test/e2e/apimachinery/webhook.go#L403-L410 and the definition for the deployment

https://github.com/kubernetes/kubernetes/blob/4e0069b9097ea19bb5af642893175dd092e64920/test/e2e/apimachinery/webhook.go#L832-L836 would suggest it fails. If that would be the case that would no longer be a "Kubernetes" cluster.

That said, I'd not mind a PR exposing the option but the real issue seems to be with the cluster.

— Reply to this email directly, view it on GitHub https://github.com/scylladb/scylla-operator/issues/962#issuecomment-1082723009, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEJSWMOEEH62PFGZMNK7SSLVCP6UJANCNFSM5R5YFRFQ . You are receiving this because you authored the thread.Message ID: @.***>