automq icon indicating copy to clipboard operation
automq copied to clipboard

perf(s3stream): prewarm MetaStream before stream ready to open

Open lifepuzzlefun opened this issue 5 months ago • 3 comments

More detailed description of your change, if necessary. The PR title and PR message become the squashed commit message, so use a separate comment to ping reviewers.

Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration tests are expected for any behaviour change and system tests should be considered for larger changes.

Committer Checklist (excluded from commit message)

  • [ ] Verify design and implementation
  • [ ] Verify test coverage and CI build status
  • [ ] Verify documentation (including upgrade notes)

lifepuzzlefun avatar Jul 28 '25 08:07 lifepuzzlefun

awaitStreamReadyForOpen often cost 500ms+, but we can trigger prewarm the first need 64KB for metastream to reduce the read time.

lifepuzzlefun avatar Jul 28 '25 08:07 lifepuzzlefun

awaitStreamReadyForOpen often cost 500ms+, but we can trigger prewarm the first need 64KB for metastream to reduce the read time.

@lifepuzzlefun The 500ms+ usually is the time cost of uploading WAL data to S3. Even if we pre-warm MetaStream, I think the MetaStream still needs to load more data from S3.

superhx avatar Jul 28 '25 08:07 superhx

This PR is being marked as stale since it has not had any activity in 90 days. If you would like to keep this PR alive, please ask a committer for review. If the PR has merge conflicts, please update it with the latest from trunk (or appropriate release branch)

If this PR is no longer valid or desired, please feel free to close it. If no activity occurs in the next 30 days, it will be automatically closed.

github-actions[bot] avatar Oct 27 '25 03:10 github-actions[bot]