autoscaling icon indicating copy to clipboard operation
autoscaling copied to clipboard

Check metrics (errors / panics / restarts / etc.) in CI

Open sharnoff opened this issue 6 months ago • 1 comments

Optional for neondatabase/company_projects#187.

Problem description / Motivation

When there's degradations that ultimately still allow e2e tests to succeed, our feedback cycle is quite slow — we only find out after cutting a release and deploying to staging (often many days after merging).

There's some clear signals we can check in CI to help with this — e.g. number of failed requests, number of panics (should be zero!), no component restarted, etc.

Feature idea(s) / DoD

This should be part of the e2e-tests workflow, mostly invisible unless it flags something.

Implementation ideas

We can probably make a small script to do this, or maybe write the shell code directly in the github action.

sharnoff avatar Jan 25 '24 18:01 sharnoff