beam icon indicating copy to clipboard operation
beam copied to clipboard

GCS client library migration in Java SDK - part 1

Open shunping opened this issue 1 month ago • 4 comments

This PR lays the groundwork for the GCS client library migration in the Java SDK. It refactors the existing GCS client implementation to prepare for a new implementation while maintaining backward compatibility.

Key changes include:

  • The current GCS client implementation has been moved to a new legacy module.
  • The existing module's public APIs have been mostly preserved and now delegate calls to the new legacy module.
  • Code and tests have been cleaned up:
    • References to the gcsio module, which will be deprecated, have been removed from GcsUtil.java.
    • Some references in GcsUtilLegacy.java and GcsOptions.java will be removed in a future PR.
  • The open API in the legacy module has been rerouted to include previously implemented but unused IO request count metrics, as discussed in https://github.com/apache/beam/pull/15394#discussion_r2552138481.

shunping avatar Nov 22 '25 05:11 shunping

cc'ed @clairemcginty

Context: gcsio is sunsetting (https://s.apache.org/beam-gcsutil-modernization), and we are on the way to migrate to the GCS client library for Beam Java SDK.

shunping avatar Nov 22 '25 13:11 shunping

Assigning reviewers:

R: @Abacn for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

github-actions[bot] avatar Nov 23 '25 00:11 github-actions[bot]

Reminder, please take a look at this pr: @Abacn

github-actions[bot] avatar Nov 30 '25 12:11 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @chamikaramj for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Dec 03 '25 12:12 github-actions[bot]

Reminder, please take a look at this pr: @chamikaramj

github-actions[bot] avatar Dec 13 '25 12:12 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @ahmedabu98 for label java.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Dec 17 '25 12:12 github-actions[bot]

waiting on author

Abacn avatar Dec 17 '25 15:12 Abacn