Oliver Gould

`"error":"received corrupt message"` makes it sound like the identity controller is starting up without the CNI being run. That is, connections from clients aren't hitting the identity controller's proxy, they...

`nsenter` is invoked here https://github.com/linkerd/linkerd2-proxy-init/blob/a556ca400132106db279ce8c3a79003a766bf707/iptables/iptables.go#L212-L228 to wrap calls to `iptables`

> The only way to set CPU and memory limits for the identity and the proxy-injector services is by hand I don't think that's correct: https://github.com/linkerd/linkerd2/blob/fe29318313005a2ebfec8d4638ec9b633b896b87/charts/linkerd-control-plane/templates/identity.yaml#L177-L179 https://github.com/linkerd/linkerd2/blob/fe29318313005a2ebfec8d4638ec9b633b896b87/charts/linkerd-control-plane/templates/proxy-injector.yaml#L95-L97

Unfortunately we had to revert the change because it broke falling back to the default control plane resources. We should have caught this in CI, but the workflow was misconfigured...

@tensor5 This sounds likely to be a cillium configuration issue? It sounds like the controllers were unable to contact the Kubernetes API Server. I'm not sure what we can change...

I took a pass at this. It's not quite there yet, but may show how we can get closer to ensuring that we setup multiple clusters: https://github.com/linkerd/linkerd2/commit/72d9b4dae590ef15b95305d7d856191ab1d96169 My basic idea...

I did some testing locally and came up with a few small suggestions https://github.com/linkerd/linkerd2/pull/9596

The screenshot you share shows the workloads having

It sounds like the identity controller is misconfigured. Is there anything in its logs? How are the issuer certificates configured via cert-manager?

I'm open to this, but we have to be sensitive to MSKV--probably pinning our client-go versions. Honestly, we probably need to do that anyway...