[Gateway API] Egress Gateways task doc is incorrect for mTLS cert verify

[SRodi]

Issue

Step 1 (verify the correct certificate of the egress gateway) in the troubleshooting section doesn't work for Gateway API.

1. New tab for Gateway API commands

2. In the new tab the following should be updated:

# current (incorrect)
$ kubectl exec -i -n istio-system "$(kubectl get pod -l istio=egressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}')" 

# should be (correct)
$ GATEWAY_NAME=cnn-egress-gateway
$ kubectl exec -i "$(kubectl get pod -l istio.io/gateway-name=$GATEWAY_NAME -o jsonpath='{.items[0].metadata.name}')" 

3. The actual instruction to verify the correct certificate of the egress gateway is not working as the cert does not seem to exist under /etc/certs/ (the whole directory does not exist)

$ kubectl exec -i "$(kubectl get pod -l istio.io/gateway-name=$GATEWAY_NAME -o jsonpath='{.items[0].metadata.name}')" -- cat /etc/certs/cert-chain.pem
cat: /etc/certs/cert-chain.pem: No such file or directory
command terminated with exit code 1

FYI @frankbu

[kfaseela]

faseelak@Faseelas-MBP istio.io % k get pods -n istio-system istio-egressgateway-684596759b-7czlt -o yaml | grep cert
      name: workload-certs
      name: istiod-ca-cert
    - mountPath: /etc/istio/egressgateway-certs
      name: egressgateway-certs
    - mountPath: /etc/istio/egressgateway-ca-certs
      name: egressgateway-ca-certs
    name: workload-certs
      name: istio-ca-root-cert
    name: istiod-ca-cert
  - name: egressgateway-certs
      secretName: istio-egressgateway-certs
  - name: egressgateway-ca-certs
      secretName: istio-egressgateway-ca-certs

[SRodi]

hi @kfaseela, thanks! I think the above is valid for Istio Egress Gateway but not applicable for Gateway API. I have updated the GH Issue description as I am referring specifically to the Gateway API instructions.

[frankbu]

@SRodi your points 1 and 2 are only needed for Gateway API, but from what @kfaseela shows in her comment, above, it sounds like the command is currently broken for Istio APIs too. IIUC, it looks like the certs are not mounted at /etc/certs (as they had been in the past), but instead at /etc/istio/egressgateway-certs, so the command to run needs to be fixed for both APIs.

[frankbu]

I talked to @howardjohn about this and he confirmed that the cert-chain.pem is gone. The following istioctl command can be used to get the SAN like this:

istioctl pc secret istio-egressgateway-5d6d8bc646-5l5pn -nistio-system -ojson | jq '[.dynamicActiveSecrets[] | select(.name == "default")][0].secret.tlsCertificate.certificateChain.inlineBytes' -r | base64 -d | openssl x509 -text -noout | grep 'Subject Alternative Name' -A 1

Step 2 in the troubleshooting section is a bigger problem, because openssl is not available in the sleep pod anymore (changed base image). John didn't think the check is really very useful anyway.

The third command might work, but not sure if it needs to change as well. It will probably only have a count of 1 (from the earlier curl), instead of 2, because we can't run the openssl call.

If it turns out that only step 1 can be fixed, we should consider removing the troubleshooting section entirely, but possibly put the step 1 command somewhere else in the doc as a tip?