beam icon indicating copy to clipboard operation
beam copied to clipboard

Add filter to GetMinUnfinishedWatermark queryAdd filter tester

Open changliiu opened this issue 7 months ago • 9 comments

To improve the performance of the query for GetUnfinishedMinWatermark, we can add a filter to the query. Note that the query run on a cache to refresh every 1s. We can remember the cached value every time and use the value as a filter in the next time query.

changliiu avatar May 23 '25 22:05 changliiu

Integration test can pass locally https://screenshot.googleplex.com/6aQBPAW8YT69VTd

changliiu avatar May 23 '25 22:05 changliiu

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

github-actions[bot] avatar May 24 '25 01:05 github-actions[bot]

Run integration test locally based on earlier commit https://github.com/apache/beam/pull/34883 and it passed. Hence the integration timeout is not caused by this PR.

changliiu avatar May 27 '25 17:05 changliiu

assign set of reviewers

changliiu avatar May 27 '25 17:05 changliiu

Assigning reviewers:

R: @m-trieu for label java. R: @nielm for label spanner.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

github-actions[bot] avatar May 27 '25 17:05 github-actions[bot]

Reminder, please take a look at this pr: @m-trieu @nielm

github-actions[bot] avatar Jun 09 '25 12:06 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @chamikaramj for label java. R: @nielm for label spanner.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Jun 11 '25 12:06 github-actions[bot]

Friendly ping :)

changliiu avatar Jun 12 '25 17:06 changliiu

Friendly ping :)

changliiu avatar Jun 13 '25 17:06 changliiu

Friendly ping :)

changliiu avatar Jun 16 '25 17:06 changliiu

It seems GCP IO integration test stucks indefinitely. Retriggered the test

update: second try still stuck, retriggering

If it timeouts for the third time, consider add class level test timeout for integraton tests that might be affected to debug

Abacn avatar Jun 18 '25 21:06 Abacn