Support serving istio and user TLS on same port

[wtzhang23]

Please provide a description of this PR: Allow serving Istio mTLS and user TLS on the same port if PeerAuthentication is permissive. See RFC.

Fixes #51768 Depends on #53396

[istio-testing]

Hi @wtzhang23. Thanks for your PR.

I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[wtzhang23]

Tested via the following script

TEST_DIR=/tmp/test-istio-dual-tls
mkdir -p ${TEST_DIR}
GOBIN=${TEST_DIR} go install istio.io/release-builder@latest

envsubst <<EOF > ${TEST_DIR}/build-manifest.yaml
version: 1.24.0
docker: docker.io/istio
directory: ${TEST_DIR}/istio-release
dependencies:
  istio:
    git: https://github.com/wtzhang23/istio
    branch: dual-tls-ingress
outputs: ["docker", "helm"]
skipGenerateBillOfMaterials: true
EOF
${TEST_DIR}/release-builder build --manifest ${TEST_DIR}/build-manifest.yaml

kind create cluster --name istio-test-dual-tls --kubeconfig ${TEST_DIR}/.kubeconfig
for archive in ${TEST_DIR}/istio-release/out/docker/*; do
    echo "Preloading ${archive}"
    kind load image-archive --name istio-test-dual-tls <(gzip --decompress --stdout ${archive})
done
kubectl create namespace --kubeconfig ${TEST_DIR}/.kubeconfig istio-system
helm install istio-base ${TEST_DIR}/istio-release/out/helm/base-1.24.0.tgz -n istio-system --kubeconfig ${TEST_DIR}/.kubeconfig
helm install istiod ${TEST_DIR}/istio-release/out/helm/istiod-1.24.0.tgz -n istio-system --kubeconfig ${TEST_DIR}/.kubeconfig --set env.ENABLE_TLS_ON_SIDECAR_INGRESS=true --wait

kubectl create namespace --kubeconfig ${TEST_DIR}/.kubeconfig test-server
kubectl label namespace --kubeconfig ${TEST_DIR}/.kubeconfig test-server istio-injection=enabled --overwrite
openssl req -x509 -newkey rsa:4096 -sha256 -days 3650 -nodes -keyout ${TEST_DIR}/tls.key -out ${TEST_DIR}/tls.crt -subj "/CN=foo.bar"
kubectl create secret tls --kubeconfig ${TEST_DIR}/.kubeconfig -n test-server tls-cert --cert ${TEST_DIR}/tls.crt --key ${TEST_DIR}/tls.key
cat <<EOF | kubectl apply --kubeconfig ${TEST_DIR}/.kubeconfig -f -
apiVersion: security.istio.io/v1
kind: PeerAuthentication
metadata:
  name: default
  namespace: test-server
spec:
  mtls:
    mode: PERMISSIVE
---
apiVersion: networking.istio.io/v1
kind: Sidecar
metadata:
  name: test-tls
  namespace: test-server
spec:
  workloadSelector:
    labels:
      app: test-server
  ingress:
    - port:
        number: 80
        protocol: HTTPS
      tls:
        mode: SIMPLE
        serverCertificate: /etc/tls-secrets/tls.crt
        privateKey: /etc/tls-secrets/tls.key
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-server
  namespace: test-server
  labels:
    app: test-server
spec:
  selector:
    matchLabels:
      app: test-server
  template:
    metadata:
      labels:
        app: test-server
      annotations:
        sidecar.istio.io/userVolumeMount: '[{"name":"tls-cert","mountPath":"/etc/tls-secrets","readOnly":true}]'
    spec:
      containers:
      - name: httpbin
        image: kennethreitz/httpbin
        ports:
        - containerPort: 80
      volumes:
      - name: tls-cert
        secret:
          secretName: tls-cert
---
apiVersion: v1
kind: Service
metadata:
  name: test-server
  namespace: test-server
spec:
  selector:
    app: test-server
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
EOF


kubectl create namespace --kubeconfig ${TEST_DIR}/.kubeconfig test-client-mesh
kubectl label namespace --kubeconfig ${TEST_DIR}/.kubeconfig test-client-mesh istio-injection=enabled --overwrite
cat <<EOF | kubectl apply --kubeconfig ${TEST_DIR}/.kubeconfig -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-client-mesh
  namespace: test-client-mesh
  labels:
    app: test-client-mesh
spec:
  selector:
    matchLabels:
      app: test-client-mesh
  template:
    metadata:
      labels:
        app: test-client-mesh
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
      volumes:
      - name: tls-cert
        secret:
          secretName: tls-cert
EOF

kubectl create namespace --kubeconfig ${TEST_DIR}/.kubeconfig test-client
cat <<EOF | kubectl apply --kubeconfig ${TEST_DIR}/.kubeconfig -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-client
  namespace: test-client
  labels:
    app: test-client
spec:
  selector:
    matchLabels:
      app: test-client
  template:
    metadata:
      labels:
        app: test-client
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
      volumes:
      - name: tls-cert
        secret:
          secretName: tls-cert
EOF

sleep 60

kubectl exec --kubeconfig ${TEST_DIR}/.kubeconfig -n test-server deploy/test-server -c istio-proxy -- curl localhost:15000/config_dump > ${TEST_DIR}/config_dump.json
kubectl exec --kubeconfig ${TEST_DIR}/.kubeconfig -n test-client-mesh deploy/test-client-mesh -- curl http://test-server.test-server.svc.cluster.local:80/headers
kubectl exec --kubeconfig ${TEST_DIR}/.kubeconfig -n test-client deploy/test-client -- curl --insecure https://test-server.test-server.svc:80/headers

[zirain]

/ok-to-test

[bleggett]

Please provide a description of this PR: Allow serving Istio mTLS and user TLS on the same port if PeerAuthentication is permissive. See RFC.

Fixes #51768 Depends on #53396

I'm not sure exactly what you mean by Istio mTLS here - there is really no such thing.

Are you saying you want the same port to terminate TLS and mTLS traffic, at the same time?

[wtzhang23]

Yes, sorry for my misleading naming. When I mention Istio mTLS, I really mean "mTLS using Istio-specific ALPNs". I'll update the wording.

[hzxuzhonghu]

To speed up the procedure, It is better to present on the meeting https://docs.google.com/document/d/1wsa06GGiq1LEGwhkiPP0FKIZJqdAiue-VeBonWAzAyk/edit#heading=h.o8pz6aqnzzgk

[wtzhang23]

Added doc to the list of those in review

[wtzhang23]

Any way to mark this as not stale?

[istio-policy-bot]

🚧 This issue or pull request has been closed due to not having had activity from an Istio team member since 2024-10-29. If you feel this issue or pull request deserves attention, please reopen the issue. Please see this wiki page for more information. Thank you for your contributions.

Created by the issue and PR lifecycle manager.