emissary
emissary copied to clipboard
emissary-ingress
`apiext` should eagerly generate TLS certificates
Please describe your use case / problem.
A user reports that they're getting timeouts of the apiserver trying to talk to apiext
Error from server: error when creating "STDIN": conversion webhook for getambassador.io/v3alpha1, Kind=Host failed: Post https://emissary-apiext.emissary-system.svc:443/webhooks/crd-convert?timeout=30s: net/http: TLS handshake timeout
despite the apiext supposedly being ready (kubectl wait --timeout=90s --for=condition=available deployment emissary-apiext -n emissary-system).
I believe that this is because apiext is using up the whole timeout lazily generating the TLS certificate. This behavior of lazily generating it originates from before we knew what the deployment story would be.
Describe the solution you'd like
Nowadays we know it always gets deployed in to the emissary-system namespace, with the same service name; it can know what hostname it needs ahead of time. And then we can build this in to the readiness checks, so a bod isn't ready until it has its TLS certificate, which would be a pretty unambiguously good thing.
Describe alternatives you've considered (none)
Additional context Ref: https://datawire-oss.slack.com/archives/CAULN7S76/p1654804207183139
Running into this fairly regularly now on 8.9.1. Cert being generated so late in the pod boot and being declared ready causes next main helm deployment to fail. Ready check hardening to wait for api-ext to configure, generate certs etc and declare itself complete would help.
Partially related #4032
We are running into similar issue and its intermittent. Is there any solution to this issue yet?
Emissary version: 3.6.0
We are running into this issue currently as well.
I have never done anything with the codebase of emissary, but I could take a look at the situation when someone points me in the right direction. But I also think that this is above my k8s knowledge level 😦
