relay icon indicating copy to clipboard operation
relay copied to clipboard

Declare Relay unhealthy when a service panics

Open jjbayer opened this issue 1 year ago • 0 comments

Following decision of internal meeting:

When a panic escapes a service loop, mark Relay's state as unhealthy such that the outside orchestrator can cut off traffic to this instance, spin up more instances and / or kill the instance.

Possible implementations:

  1. Introduce join handles and let the main task flip the health state to unhealthy once a join handle returns with a panic.
  2. Force every service implementation to report a health state, report overall unhealthy as soon as a single service reports unhealthy (OR condiditon
  3. ... ?

ref: INC-875

jjbayer avatar Sep 16 '24 11:09 jjbayer