helm-charts icon indicating copy to clipboard operation
helm-charts copied to clipboard

Add readiness probes

Open mtparet opened this issue 5 months ago • 2 comments

What was changed

Adding readiness probes to web and server deployment.

Why?

Currently we send requests to these pods even when these are not fully started. These basic readiness should help wait before sending requests that temporal is up.

Checklist

  1. Closes #710

mtparet avatar Jun 05 '25 09:06 mtparet

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Jun 05 '25 09:06 CLAassistant

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

CLAassistant avatar Jun 05 '25 09:06 CLAassistant

Hey team, Is this going to be reviewed/merged anytime soon?

We experiencing drops anytime we upgrade Temporal cluster with multiple replicas. While we're following RollingUpdate strategy and rolling out performed 1 by 1, I've noticed that pod immediately reaches Ready status while logs still states that: "operation":"PollActivityTaskQueue","wf-namespace":"<redacted>","error":"Frontend is not healthy yet"

And only several seconds (or faster) later Frontend is now healthy

Under the heavy load we encountering too many rejects which affecting performance when using Temporal

Thank you!

VLZZZ avatar Aug 01 '25 12:08 VLZZZ