beam icon indicating copy to clipboard operation
beam copied to clipboard

[DRAFT] Attempt fix Jms watermark

Open Abacn opened this issue 1 year ago • 1 comments

Originally, the problem was watermark set to Instant.now() at initialization and after finalizing every non zero checkpoint mark. This had two issues

  • watermark can be too new for incoming data, when checkpoint is not finalized
  • watermark does not increase when there is no incoming data for a while

This PR attempted to address both.

Please add a meaningful description for your change here


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • [ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • [ ] Update CHANGES.md with noteworthy changes.
  • [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels Python tests Java tests Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Abacn avatar Feb 16 '24 20:02 Abacn

However, it seems currently it's not work as expected. The watermark still increase when no incoming data:

image

Abacn avatar Feb 16 '24 20:02 Abacn

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

github-actions[bot] avatar Apr 17 '24 12:04 github-actions[bot]

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

github-actions[bot] avatar Apr 24 '24 12:04 github-actions[bot]