beam icon indicating copy to clipboard operation
beam copied to clipboard

[DO NOT MERGE] Run all PostCommit and PreCommit Tests against Release Branch

Open Amar3tto opened this issue 6 months ago • 2 comments

Tests run on GHA should be queued shortly.

Amar3tto avatar Jun 13 '25 13:06 Amar3tto

Test Results

15 tests   15 ✅  1h 46m 28s ⏱️ 12 suites   0 💤 12 files     0 ❌

Results for commit f1935a79.

github-actions[bot] avatar Jun 13 '25 14:06 github-actions[bot]

Assigning reviewers:

R: @damccorm for label build.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

github-actions[bot] avatar Jun 13 '25 15:06 github-actions[bot]

@ahmedabu98 Could you please take a look at the failed Iceberg checks? I'm not sure why they fail here and succeed on master.

Amar3tto avatar Jun 16 '25 14:06 Amar3tto

@Amar3tto I checked the Dataflow pipelines. The tests on master are pulling the SDK container image from --sdk_harness_ids=sdk-0-0,gcr.io/cloud-dataflow/v1beta3/beam_java11_sdk:beam-master-20250605 and passing successfully.

Dataflow tests on this PR are pulling from -sdk_harness_ids=sdk-0-0,gcr.io/cloud-dataflow/v1beta3/beam_java11_sdk:2.66.0 and failing with this error:

"PullImage from image service failed" 
err="rpc error: code = NotFound desc = failed to pull and unpack image 
  \"gcr.io/cloud-dataflow/v1beta3/beam_java11_sdk:2.66.0\": 
  failed to resolve reference \"gcr.io/cloud-dataflow/v1beta3/beam_java11_sdk:2.66.0\": 
  gcr.io/cloud-dataflow/v1beta3/beam_java11_sdk:2.66.0: not found" 
image="gcr.io/cloud-dataflow/v1beta3/beam_java11_sdk:2.66.0"

The difference reflects this change: https://github.com/apache/beam/commit/086ae6cf8311799c1d7ee2eac5eb3adff9253e30

My guess is the Dataflow container just hasn't been built and uploaded to that location yet

ahmedabu98 avatar Jun 16 '25 17:06 ahmedabu98

Yes, see - https://github.com/apache/beam/blob/master/contributor-docs/release-guide.md#verify-release-branch

"NOTE Dataflow tests will fail if the Dataflow worker container is not created and published by this time. Should be done by Google, in response to the creation of the release branch, and docker images are hosted. This should not block creation of the first release candidate, but should block approval of the release."

damccorm avatar Jun 16 '25 17:06 damccorm

@Amar3tto I'd recommend that you go ahead and create the first RC. At that point, I can publish the Dataflow containers needed to make this pass.

damccorm avatar Jun 16 '25 17:06 damccorm

Hmmm I see that the other Dataflow tests are passing because they first build and publish a new container image to use for the test, e.g.: https://github.com/apache/beam/blob/19ff1dfd71493341c5f78581f033ba11190cb74f/runners/google-cloud-dataflow-java/build.gradle#L712-L714

Iceberg Dataflow tests are missing this step.

ahmedabu98 avatar Jun 16 '25 17:06 ahmedabu98

Adding it in #35307

ahmedabu98 avatar Jun 16 '25 17:06 ahmedabu98

Reminder, please take a look at this pr: @damccorm

github-actions[bot] avatar Jun 24 '25 12:06 github-actions[bot]

stop reviewer notifications

damccorm avatar Jun 24 '25 13:06 damccorm

Stopping reviewer notifications for this pull request: requested by reviewer. If you'd like to restart, comment assign set of reviewers

github-actions[bot] avatar Jun 24 '25 13:06 github-actions[bot]