monogon icon indicating copy to clipboard operation
monogon copied to clipboard

Published 20 hours ago • verified publisher icon monogon-dev

Metropolis Control Plane HA

Open q3k opened this issue 2 years ago • 1 comments

We've never really tested Control Plane HA in an end-to-end scenario, and we reaped the effects of that during our first large production deploying.

Related fixes:

  1. https://review.monogon.dev/c/monogon/+/2067
  2. https://review.monogon.dev/c/monogon/+/2068
  3. https://review.monogon.dev/c/monogon/+/2069
  4. https://review.monogon.dev/c/monogon/+/2071

E2E tests are still in progress.

q3k avatar Aug 22 '23 12:08 q3k

Forensics data from a recent HA control plane failure:

https://drive.google.com/drive/folders/1knvTErYqXKEXUyCxvaNpHyM4tikz7QXY

q3k avatar Sep 05 '23 12:09 q3k

Our assumption is that this fixes that, right? https://review.monogon.dev/c/monogon/+/2873

leoluk avatar Aug 10 '24 17:08 leoluk

Closing unless proven otherwise - discussed with Lorenz and we're not aware of any outstanding issues (beyond more QA burn-in time).

leoluk avatar Aug 10 '24 17:08 leoluk