bug: the net-istio documentation is outdated in the case of the knative operator.

[metacoma]

Expected Behavior

net-istio components works as expected

Actual Behavior

After applying the knative serving resource using the knative operator and deploying the net-istio components, two replicasets are created for the net-istio controller and net-istio webhook components.

Steps to Reproduce the Problem

1. Install knative-operator
2. Install Serving resource in knative-serving namespace
3. Install istio network layer
4. kubectl -n knative-serving get replicaset

$ kubectl -n knative-serving get replicaset -l app=net-istio-webhook
NAME                           DESIRED   CURRENT   READY   AGE
net-istio-webhook-5fd875d5cb   1         1         1       13d
net-istio-webhook-6b474755fb   0         0         0       13d
$ kubectl -n knative-serving get replicaset -l app=net-istio-controller
NAME                              DESIRED   CURRENT   READY   AGE
net-istio-controller-55c49d86cc   0         0         0       13d
net-istio-controller-6489dd6f5b   1         1         1       13d

5. The net-istio controller and net-istio webhook pods restart every 3 minutes.

$ kubectl -n knative-serving get pods -w
net-istio-controller-55c49d86cc-pg4vt                    0/1     Pending     0               0s
net-istio-webhook-6b474755fb-d5nlj                       0/2     Pending     0               0s
net-istio-controller-55c49d86cc-pg4vt                    0/1     Pending     0               0s
net-istio-webhook-6b474755fb-d5nlj                       0/2     Pending     0               0s
net-istio-controller-55c49d86cc-pg4vt                    0/1     ContainerCreating   0               0s
net-istio-webhook-6b474755fb-d5nlj                       0/2     Init:0/1            0               0s
net-istio-controller-55c49d86cc-pg4vt                    0/1     Running             0               2s
net-istio-webhook-6b474755fb-d5nlj                       0/2     PodInitializing     0               2s
net-istio-webhook-6b474755fb-d5nlj                       0/2     Running             0               3s
net-istio-controller-55c49d86cc-pg4vt                    1/1     Running             0               3s
net-istio-controller-6489dd6f5b-d4sr8                    1/1     Terminating         0               2m51s
net-istio-controller-6489dd6f5b-d4sr8                    0/1     Completed           0               2m52s
net-istio-webhook-6b474755fb-d5nlj                       0/2     Running             0               4s
net-istio-webhook-6b474755fb-d5nlj                       1/2     Running             0               4s
net-istio-webhook-6b474755fb-d5nlj                       2/2     Running             0               4s
net-istio-controller-6489dd6f5b-d4sr8                    0/1     Completed           0               2m52s
net-istio-controller-6489dd6f5b-d4sr8                    0/1     Completed           0               2m52s
net-istio-webhook-5fd875d5cb-nsrkv                       1/1     Terminating         0               2m52s
net-istio-webhook-5fd875d5cb-nsrkv                       0/1     Terminating         0               2m55s

Additional Info

Additional context As I understand it, this bug was introduced in the 1.14 release. For some reason, Net-Istio was included in the Knative Operator codebase. https://github.com/knative/operator/blob/a0cc535e3eaa464b0b28a4dea0b3a614093829be/cmd/operator/kodata/ingress/1.14/istio/net-istio.yaml

Install information:

$ kubectl version
Client Version: v1.31.0
Kustomize Version: v5.4.2
Server Version: v1.32.0+k3s1
knative-eventing   1.16.0    True   
knative-serving   1.16.0    True    
knative-operator: 1.16.0 

apiVersion: v1
items:
- apiVersion: operator.knative.dev/v1beta1
  kind: KnativeServing
  ...
  spec:
    config:
      deployment:
        registries-skipping-tag-resolving: zot-int.zot.svc.cluster.local:5000,zot-int.zot:5000,127.0.0.1:30001
      observability:
        metrics.backend-destination: prometheus
        metrics.request-metrics-backend-destination: prometheus
      tracing:
        backend: zipkin
        debug: "true"
        sample-rate: "1"
        zipkin-endpoint: http://tempo.monitoring:9411/api/v2/spans
    namespace:
      labels:
        istio-injection: enabled
```

[dprotaso]

Install istio network layer

So the operator will install the net-istio plugin - but we expect you to install Istio.

Does it mention in the docs somewhere to install both?

[metacoma]

hey @dprotaso sorry for confusing

TLDR:

1. A long time ago, I added knative-operator 1.13 in my gitops repository. According to the documentation, I had to add net-istio.yaml as a networklayer for integration with Itio

2. A few weeks ago, I wanted to upgrade Knative Operator to 1.17 in gitops repo. This led to a conflict between net-istio from the Knative Operator release and the net-istio that was previously installed from step 1, according to the documentation.

I created this issue because I spent two days investigating the circumstances, and when I finally identified the root cause, I started searching for mentions of this in the changelogs, repository, or Slack, but I found nothing.

As far as I understand, there are no e2e tests for upgrading procedure, which is probably the reason for this issue.

Feel free to close this issue if it is irrelevant.

[dprotaso]

Operator always installed net-istio

You can see v1.13 had the manifest here - https://github.com/knative/operator/tree/release-1.15/cmd/operator/kodata/ingress/1.13/istio

[metacoma]

Operator always installed net-istio

Ohh, indeed, I double-checked the branches, and it's there.

In this case, should this information be mentioned somewhere here: https://knative.dev/docs/install/installing-istio/#before-you-begin ?

[dprotaso]

Yeah I think that makes sense

[dprotaso]

/triage accepted