linkerd2 icon indicating copy to clipboard operation
linkerd2 copied to clipboard

Increase Mutating webhook timeout.

Open alpeb opened this issue 1 year ago • 3 comments

Discussed in https://github.com/linkerd/linkerd2/discussions/11984

Originally posted by smithdebug January 24, 2024 Hello Team, I’m reaching out to inquire about the possibility of extending the timeout for the linkerd-proxy-injector-webhook-config Mutating Webhook. Currently, the timeout is set to 10 seconds bydefault. We’ve recently faced a situation where this timeout caused an issue, resulting in a Pod without a Linkerd proxy sidecar. Thanks

alpeb avatar Jan 29 '24 22:01 alpeb

@smithdebug , would you be willing to tackle this issue? Sounds like a relatively simple setting to add and hook in the helm templates.

alpeb avatar Feb 01 '24 16:02 alpeb

Thanks for opening #12028, we'll take a look at it soon. OTOH, we're still interested on why you actually bumped into this issue. The proxy injector should not take that long to process pods. Can you share a bit more about your setup? Is there a lot of pod churn, maybe during node escalations, causing requests to the injector to spike? Are you already running in HA mode, increasing the number of the injector replicas?

alpeb avatar Feb 08 '24 18:02 alpeb

We have around 3000 pods and 5 replicas for proxy-injector. We have enabled webhook failure mode, earlier it was set to ignore.

smithdebug avatar Feb 15 '24 00:02 smithdebug

Is this issue still relevant?

wim-de-groot avatar Mar 27 '24 05:03 wim-de-groot

@wim-de-groot nope! thanks for pointing it out.

Closing, since this has been resolved through https://github.com/linkerd/linkerd2/pull/12028. Thanks all!

mateiidavid avatar Mar 27 '24 11:03 mateiidavid