beam icon indicating copy to clipboard operation
beam copied to clipboard

[Java]Fix Neo4jIO WriteUnwindWithResultsFn for Statistics Emission

Open Suvrat1629 opened this issue 4 months ago • 21 comments

Please add a meaningful description for your change here

This PR addresses #33041, a blocker for Neo4jIO in the Neo4j Flex template. It fixes WriteUnwindWithResultsFn to emit Neo4jWriteStats, resolving compilation errors. Further work is needed to enhance WriteUnwind and improve timestamp accuracy.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • [x] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • [ ] Update CHANGES.md with noteworthy changes.
  • [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels Python tests Java tests Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Suvrat1629 avatar Aug 24 '25 19:08 Suvrat1629

@fbiville I have taken an attempt at getting started with your issue(#33041) would be nice if you could guide me for improving this pr further.

These are the things I need help addressing:

  • The original WriteUnwind transform remains a sink (returning PDone), which is the primary blocker for the Flex template. It needs to be enhanced to return a PCollection<Neo4jWriteStats> or deprecated in favor of WriteUnwindWithResults.
  • Timestamp accuracy needs improvement (e.g., using ValueState) to ensure proper window alignment with Wait.on in Dataflow.

Suvrat1629 avatar Aug 24 '25 19:08 Suvrat1629

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

github-actions[bot] avatar Aug 24 '25 20:08 github-actions[bot]

assign set of reviewers

Suvrat1629 avatar Aug 29 '25 18:08 Suvrat1629

Assigning reviewers:

R: @Abacn for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

github-actions[bot] avatar Aug 29 '25 18:08 github-actions[bot]

Reminder, please take a look at this pr: @Abacn

github-actions[bot] avatar Sep 06 '25 12:09 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @robertwb for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Sep 10 '25 12:09 github-actions[bot]

Reminder, please take a look at this pr: @robertwb

github-actions[bot] avatar Sep 17 '25 12:09 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Sep 22 '25 12:09 github-actions[bot]

Reminder, please take a look at this pr: @kennknowles

github-actions[bot] avatar Sep 29 '25 12:09 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @robertwb for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Oct 02 '25 12:10 github-actions[bot]

Reminder, please take a look at this pr: @robertwb

github-actions[bot] avatar Oct 09 '25 12:10 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Oct 13 '25 12:10 github-actions[bot]

Reminder, please take a look at this pr: @kennknowles

github-actions[bot] avatar Oct 24 '25 12:10 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @chamikaramj for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Oct 29 '25 12:10 github-actions[bot]

Reminder, please take a look at this pr: @chamikaramj

github-actions[bot] avatar Nov 05 '25 12:11 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Nov 10 '25 12:11 github-actions[bot]

Reminder, please take a look at this pr: @Abacn

github-actions[bot] avatar Nov 18 '25 12:11 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @chamikaramj for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Nov 21 '25 12:11 github-actions[bot]

Reminder, please take a look at this pr: @chamikaramj

github-actions[bot] avatar Nov 28 '25 12:11 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @ahmedabu98 for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Dec 02 '25 12:12 github-actions[bot]

Reminder, please take a look at this pr: @ahmedabu98

github-actions[bot] avatar Dec 10 '25 12:12 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Dec 15 '25 12:12 github-actions[bot]