More resilient startup

Open paulfantom opened this issue 3 years ago • 1 comments

Currently when promscale cannot connect to database it almost immediately crashes. In environments where database is started at the same time as promscale (like tobs) this can lead to multiple Crash Loops. Since kubernetes does an exponential crashloop backoff, it leads to long delay in startup.

My proposal is to add a timeout at the startup to check for connection for about 120s (ideally this should be customizable via cli flag). This way long database startup won't affect the whole stack startup that much.

pinging @VineethReddy02 @cevian for task prioritization after release.

May 11 '22 10:05 paulfantom

duplicate to #592

May 30 '22 11:05 VineethReddy02

closing this issue, as the same issue is created by the user in #592

Aug 17 '22 06:08 VineethReddy02