posthog icon indicating copy to clipboard operation
posthog copied to clipboard

feat(capture): expose Hyper server loop directly to control/observe graceful shutdown

Open eli-r-ph opened this issue 1 month ago • 1 comments

Problem

We want more app level control over graceful shutdown of capture's Axum server during deploy cutovers.

Changes

  • Expose underlying Hyper server builder plumbing
  • Control graceful shutdown timeout and observe actual behavior of Hyper-managed connections on shutdown

How did you test this code?

Locally, CI, mirror deploy smoke test

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Changelog: (features only) Is this feature complete?

N/A

eli-r-ph avatar Dec 09 '25 22:12 eli-r-ph

Wiz Scan Summary

Scanner Findings
Vulnerability Finding Vulnerabilities 3 High 10 Medium 2 Low
Data Finding Sensitive Data -
Secret Finding Secrets -
IaC Misconfiguration IaC Misconfigurations -
SAST Finding SAST Findings -
Total 3 High 10 Medium 2 Low

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.

wiz-7ad640923b[bot] avatar Dec 09 '25 22:12 wiz-7ad640923b[bot]

Tested in mirror today. Sadly, none of the new metrics or logs emitted on shutdown. I'm looking into why (I assume final metrics/log flush isn't making it out as the shutdown handler is triggered very late in the game but still digging.)

This change could still be useful if we need control to drain queued connections prior to shutdown to solve the root cause of the conn closed issue w/Envoy definitively. In the meantime, I'm putting this back in draft and testing the PR I expect to actually fix that problem (correcting _readiness probe behavior)

eli-r-ph avatar Dec 10 '25 23:12 eli-r-ph

Closing this in favor of more robust, rebased impl here

eli-r-ph avatar Dec 13 '25 00:12 eli-r-ph